[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:29:22 dist-test-slave-4qw6 krb5kdc[17802](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:29:22 dist-test-slave-4qw6 krb5kdc[17802](info): set up 2 sockets
Jun 19 02:29:22 dist-test-slave-4qw6 krb5kdc[17802](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260619 02:29:23.131119 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:29:23 dist-test-slave-4qw6 krb5kdc[17802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:29:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:29:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:29:23.277943 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:34043
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:34123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:34043
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:23.398980 17815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:23.399323 17815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:23.399367 17815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:23.403954 17815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:29:23.404031 17815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:23.404052 17815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:23.404073 17815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:29:23.404090 17815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:29:23.410108 17815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:34123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:34043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:34043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:23.411369 17815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:23.412518 17815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:23.422323 17821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:23.422323 17820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:23.422394 17815 server_base.cc:1034] running on GCE node
W20260619 02:29:23.422340 17823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:23.423195 17815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:23.424571 17815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:23.425783 17815 hybrid_clock.cc:648] HybridClock initialized: now 1781836163425731 us; error 31 us; skew 500 ppm
Jun 19 02:29:23 dist-test-slave-4qw6 krb5kdc[17802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836163, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:23.428783 17815 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:29:23.430011 17815 webserver.cc:460] Webserver started at http://127.17.96.254:44623/ using document root <none> and password file <none>
I20260619 02:29:23.430692 17815 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:23.430766 17815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:23.431031 17815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:23.433436 17815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "d5a1981c1a8f470c8655a34e01053ec6"
format_stamp: "Formatted at 2026-06-19 02:29:23 on dist-test-slave-4qw6"
server_key: "8ca37e79617e9d92ddcbaa7e8430469e"
server_key_iv: "a3eb1d24ee041a6f912ff09f24c35404"
server_key_version: "encryptionkey@0"
I20260619 02:29:23.434156 17815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "d5a1981c1a8f470c8655a34e01053ec6"
format_stamp: "Formatted at 2026-06-19 02:29:23 on dist-test-slave-4qw6"
server_key: "8ca37e79617e9d92ddcbaa7e8430469e"
server_key_iv: "a3eb1d24ee041a6f912ff09f24c35404"
server_key_version: "encryptionkey@0"
I20260619 02:29:23.438656 17815 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 02:29:23.441504 17830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:23.443089 17815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 02:29:23.443279 17815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "d5a1981c1a8f470c8655a34e01053ec6"
format_stamp: "Formatted at 2026-06-19 02:29:23 on dist-test-slave-4qw6"
server_key: "8ca37e79617e9d92ddcbaa7e8430469e"
server_key_iv: "a3eb1d24ee041a6f912ff09f24c35404"
server_key_version: "encryptionkey@0"
I20260619 02:29:23.443428 17815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:23.459391 17815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:23.462906 17815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:23.463151 17815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:23.470759 17815 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:34043
I20260619 02:29:23.470835 17882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:34043 every 8 connection(s)
I20260619 02:29:23.471940 17815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:29:23.475304 17883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:23.475731 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 17815
I20260619 02:29:23.475914 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:29:23.477644 17795 external_mini_cluster.cc:1428] Setting key a68954534b54b7b8f7e18054ae1a6cb4
I20260619 02:29:23.483218 17883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5a1981c1a8f470c8655a34e01053ec6: Bootstrap starting.
I20260619 02:29:23.486577 17883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5a1981c1a8f470c8655a34e01053ec6: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:23.487687 17883 log.cc:824] T 00000000000000000000000000000000 P d5a1981c1a8f470c8655a34e01053ec6: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:23.490486 17883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5a1981c1a8f470c8655a34e01053ec6: No bootstrap required, opened a new log
I20260619 02:29:23.495361 17883 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5a1981c1a8f470c8655a34e01053ec6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a1981c1a8f470c8655a34e01053ec6" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34043 } }
I20260619 02:29:23.495735 17883 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5a1981c1a8f470c8655a34e01053ec6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:23.495831 17883 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5a1981c1a8f470c8655a34e01053ec6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a1981c1a8f470c8655a34e01053ec6, State: Initialized, Role: FOLLOWER
I20260619 02:29:23.496372 17883 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5a1981c1a8f470c8655a34e01053ec6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a1981c1a8f470c8655a34e01053ec6" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34043 } }
I20260619 02:29:23.496572 17883 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5a1981c1a8f470c8655a34e01053ec6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:29:23.496649 17883 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5a1981c1a8f470c8655a34e01053ec6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:29:23.496747 17883 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5a1981c1a8f470c8655a34e01053ec6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:23.498200 17883 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5a1981c1a8f470c8655a34e01053ec6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a1981c1a8f470c8655a34e01053ec6" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34043 } }
I20260619 02:29:23.498656 17883 leader_election.cc:302] T 00000000000000000000000000000000 P d5a1981c1a8f470c8655a34e01053ec6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5a1981c1a8f470c8655a34e01053ec6; no voters:
I20260619 02:29:23.499078 17883 leader_election.cc:288] T 00000000000000000000000000000000 P d5a1981c1a8f470c8655a34e01053ec6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:29:23.499195 17888 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5a1981c1a8f470c8655a34e01053ec6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:23.499485 17888 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5a1981c1a8f470c8655a34e01053ec6 [term 1 LEADER]: Becoming Leader. State: Replica: d5a1981c1a8f470c8655a34e01053ec6, State: Running, Role: LEADER
I20260619 02:29:23.499833 17888 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5a1981c1a8f470c8655a34e01053ec6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a1981c1a8f470c8655a34e01053ec6" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34043 } }
I20260619 02:29:23.500612 17883 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5a1981c1a8f470c8655a34e01053ec6 [sys.catalog]: configured and running, proceeding with master startup.
Jun 19 02:29:23 dist-test-slave-4qw6 krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:23.501947 17890 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5a1981c1a8f470c8655a34e01053ec6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5a1981c1a8f470c8655a34e01053ec6. Latest consensus state: current_term: 1 leader_uuid: "d5a1981c1a8f470c8655a34e01053ec6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a1981c1a8f470c8655a34e01053ec6" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34043 } } }
I20260619 02:29:23.501947 17889 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5a1981c1a8f470c8655a34e01053ec6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5a1981c1a8f470c8655a34e01053ec6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a1981c1a8f470c8655a34e01053ec6" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34043 } } }
I20260619 02:29:23.502108 17889 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5a1981c1a8f470c8655a34e01053ec6 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:23.502108 17890 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5a1981c1a8f470c8655a34e01053ec6 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:23.502576 17897 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:29:23.506326 17897 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:29:23.508883 17885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:23.481937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55036 (local address 127.17.96.254:34043)
0619 02:29:23.482524 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:23.482545 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:23.483605 (+ 1060us) server_negotiation.cc:407] Connection header received
0619 02:29:23.488534 (+ 4929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:23.488563 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:23.489012 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:23.489344 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:23.492779 (+ 3435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:23.493921 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:23.495723 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:23.496332 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:23.496470 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:23.501955 (+ 5485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:23.501964 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:23.501997 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:23.502044 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:23.504318 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:23.505097 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:23.505104 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:23.505110 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:23.505258 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:23.505721 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:23.505725 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:23.505727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:23.506305 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:23.506543 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:23.507452 (+ 909us) server_negotiation.cc:299] Negotiation successful
0619 02:29:23.507807 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":169,"threads_started":1}
I20260619 02:29:23.513330 17897 catalog_manager.cc:1347] Generated new cluster ID: de31cf1e69354970a4b855fd4919fc9d
I20260619 02:29:23.513437 17897 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:29:23.536643 17897 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:29:23.538141 17897 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:29:23.548184 17897 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5a1981c1a8f470c8655a34e01053ec6: Generated new TSK 0
I20260619 02:29:23.549125 17897 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.96.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
I20260619 02:29:23.583323 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:34043
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:34123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:23.711884 17909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:23.712277 17909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:23.712368 17909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:23.716959 17909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:23.717121 17909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:23.717239 17909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:29:23.723938 17909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:34123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:34043
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:23.725387 17909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:23.726708 17909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:23.738214 17914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:23.738504 17909 server_base.cc:1034] running on GCE node
W20260619 02:29:23.738219 17917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:23.738394 17915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:23.739279 17909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:23.740331 17909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:23.741595 17909 hybrid_clock.cc:648] HybridClock initialized: now 1781836163741561 us; error 58 us; skew 500 ppm
Jun 19 02:29:23 dist-test-slave-4qw6 krb5kdc[17802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836163, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:23.745488 17909 init.cc:377] Logged in from keytab as kudu/127.17.96.193@KRBTEST.COM (short username kudu)
I20260619 02:29:23.746768 17909 webserver.cc:460] Webserver started at http://127.17.96.193:42551/ using document root <none> and password file <none>
I20260619 02:29:23.747596 17909 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:23.747714 17909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:23.748021 17909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:23.750739 17909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "9a4be293ca4e4497957506401a27a6a7"
format_stamp: "Formatted at 2026-06-19 02:29:23 on dist-test-slave-4qw6"
server_key: "c36c5565ac487113bafd732c76d22fd0"
server_key_iv: "f84bb16b70adb7f0f0d4a6e2bd22e5f6"
server_key_version: "encryptionkey@0"
I20260619 02:29:23.751459 17909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "9a4be293ca4e4497957506401a27a6a7"
format_stamp: "Formatted at 2026-06-19 02:29:23 on dist-test-slave-4qw6"
server_key: "c36c5565ac487113bafd732c76d22fd0"
server_key_iv: "f84bb16b70adb7f0f0d4a6e2bd22e5f6"
server_key_version: "encryptionkey@0"
I20260619 02:29:23.756450 17909 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:29:23.760663 17924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:23.762833 17909 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260619 02:29:23.763048 17909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "9a4be293ca4e4497957506401a27a6a7"
format_stamp: "Formatted at 2026-06-19 02:29:23 on dist-test-slave-4qw6"
server_key: "c36c5565ac487113bafd732c76d22fd0"
server_key_iv: "f84bb16b70adb7f0f0d4a6e2bd22e5f6"
server_key_version: "encryptionkey@0"
I20260619 02:29:23.763247 17909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:23.790642 17909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:23.794198 17909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:23.794432 17909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:23.795106 17909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:23.796025 17909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:23.796073 17909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:23.796149 17909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:23.796180 17909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:23.806047 17909 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:42769
I20260619 02:29:23.806118 18037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:42769 every 8 connection(s)
I20260619 02:29:23.807068 17909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:29:23.812161 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 17909
I20260619 02:29:23.812268 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:29:23.812574 17795 external_mini_cluster.cc:1428] Setting key e9467f4f86625b3990d759065cf805fa
Jun 19 02:29:23 dist-test-slave-4qw6 krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836163, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.96.194@KRBTEST.COM; defaulting to no policy
I20260619 02:29:23.823047 17885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:23.809122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:35593 (local address 127.17.96.254:34043)
0619 02:29:23.809309 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:23.809314 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:23.810292 (+ 978us) server_negotiation.cc:407] Connection header received
0619 02:29:23.811668 (+ 1376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:23.811672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:23.811726 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:23.811870 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:23.813472 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:23.814156 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:23.815467 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:23.815779 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:23.815916 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:23.818927 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:23.818931 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:23.818933 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:23.818962 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:23.820561 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:23.821177 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:23.821181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:23.821183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:23.821272 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:23.821650 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:23.821655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:23.821656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:23.821935 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:23.822051 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:23.822738 (+ 687us) server_negotiation.cc:299] Negotiation successful
0619 02:29:23.822903 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 02:29:23.823946 18040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:23.809579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:34043 (local address 127.17.96.193:35593)
0619 02:29:23.810119 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:23.810167 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:23.811426 (+ 1259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:23.812079 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:23.812087 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:23.812582 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:23.813306 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:23.813327 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:23.814319 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:23.814323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:23.815313 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:23.815325 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:23.815987 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:23.815992 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:23.816166 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:23.816881 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:23.816899 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:23.818710 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:23.820722 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:23.820730 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:23.820750 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:23.821054 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:23.821378 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:23.821381 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:23.821384 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:23.821547 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:23.822049 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:23.822058 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:23.822357 (+ 299us) client_negotiation.cc:769] Sending connection context
0619 02:29:23.822617 (+ 260us) client_negotiation.cc:240] Negotiation successful
0619 02:29:23.822929 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":150,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.194@KRBTEST.COM" created.
I20260619 02:29:23.825471 18038 heartbeater.cc:347] Connected to a master server at 127.17.96.254:34043
I20260619 02:29:23.825738 18038 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:23.826347 18038 heartbeater.cc:511] Master 127.17.96.254:34043 requested a full tablet report, sending...
I20260619 02:29:23.828110 17847 ts_manager.cc:194] Registered new tserver with Master: 9a4be293ca4e4497957506401a27a6a7 (127.17.96.193:42769)
I20260619 02:29:23.829452 17847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:35593
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
I20260619 02:29:23.841181 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:34043
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:34123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:23.959808 18043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:23.960146 18043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:23.960196 18043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:23.964689 18043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:23.964777 18043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:23.964846 18043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:29:23.970842 18043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:34123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:34043
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:23.972087 18043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:23.973223 18043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:23.983909 18048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:23.983913 18049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:23.983884 18051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:23.984369 18043 server_base.cc:1034] running on GCE node
I20260619 02:29:23.984949 18043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:23.985695 18043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:23.986923 18043 hybrid_clock.cc:648] HybridClock initialized: now 1781836163986904 us; error 33 us; skew 500 ppm
Jun 19 02:29:23 dist-test-slave-4qw6 krb5kdc[17802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836163, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:23.989876 18043 init.cc:377] Logged in from keytab as kudu/127.17.96.194@KRBTEST.COM (short username kudu)
I20260619 02:29:23.990887 18043 webserver.cc:460] Webserver started at http://127.17.96.194:39155/ using document root <none> and password file <none>
I20260619 02:29:23.991518 18043 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:23.991616 18043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:23.991866 18043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:23.994210 18043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "80001c82d6e640a4899a76d368ffc28b"
format_stamp: "Formatted at 2026-06-19 02:29:23 on dist-test-slave-4qw6"
server_key: "19df43103fa1e59a6a45ac0a739aa60c"
server_key_iv: "45d0f34fc9817638fb38954f710ffc89"
server_key_version: "encryptionkey@0"
I20260619 02:29:23.994840 18043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "80001c82d6e640a4899a76d368ffc28b"
format_stamp: "Formatted at 2026-06-19 02:29:23 on dist-test-slave-4qw6"
server_key: "19df43103fa1e59a6a45ac0a739aa60c"
server_key_iv: "45d0f34fc9817638fb38954f710ffc89"
server_key_version: "encryptionkey@0"
I20260619 02:29:23.999116 18043 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 02:29:24.002242 18058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:24.003820 18043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:29:24.004004 18043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "80001c82d6e640a4899a76d368ffc28b"
format_stamp: "Formatted at 2026-06-19 02:29:23 on dist-test-slave-4qw6"
server_key: "19df43103fa1e59a6a45ac0a739aa60c"
server_key_iv: "45d0f34fc9817638fb38954f710ffc89"
server_key_version: "encryptionkey@0"
I20260619 02:29:24.004154 18043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:24.028239 18043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:24.031963 18043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:24.032207 18043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:24.032888 18043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:24.033926 18043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:24.034008 18043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:24.034099 18043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:24.034152 18043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:24.043941 18043 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:32889
I20260619 02:29:24.043964 18171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:32889 every 8 connection(s)
I20260619 02:29:24.045001 18043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:29:24.048544 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 18043
I20260619 02:29:24.048681 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:29:24.049016 17795 external_mini_cluster.cc:1428] Setting key 33f5693a158bcfb0406f862059b08c26
Jun 19 02:29:24 dist-test-slave-4qw6 krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836163, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.96.195@KRBTEST.COM; defaulting to no policy
I20260619 02:29:24.061129 17885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:24.047029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:57233 (local address 127.17.96.254:34043)
0619 02:29:24.047222 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:24.047228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:24.048109 (+ 881us) server_negotiation.cc:407] Connection header received
0619 02:29:24.049945 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:24.049949 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:24.050012 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:24.050129 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:24.051900 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:24.052588 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:24.053873 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:24.054109 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:24.054243 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:24.057017 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:24.057021 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:24.057023 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:24.057052 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:24.058700 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:24.059360 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:24.059363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:24.059367 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:24.059458 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:24.059845 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:24.059850 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:24.059852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:24.060124 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:24.060237 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:24.060794 (+ 557us) server_negotiation.cc:299] Negotiation successful
0619 02:29:24.060951 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.195@KRBTEST.COM" created.
I20260619 02:29:24.062119 18174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:24.047431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:34043 (local address 127.17.96.194:57233)
0619 02:29:24.047959 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:24.048008 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:24.049704 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:24.050321 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:24.050329 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:24.050919 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:24.051728 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:24.051751 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:24.052740 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:24.052744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:24.053681 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:24.053691 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:24.054292 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:24.054296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:24.054508 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:24.055191 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:24.055210 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:24.056806 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:24.058848 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:24.058859 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:24.058875 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:24.059238 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:24.059571 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:24.059575 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:24.059577 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:24.059744 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:24.060234 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:24.060242 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:24.060541 (+ 299us) client_negotiation.cc:769] Sending connection context
0619 02:29:24.060751 (+ 210us) client_negotiation.cc:240] Negotiation successful
0619 02:29:24.061143 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":123,"threads_started":1}
I20260619 02:29:24.063470 18172 heartbeater.cc:347] Connected to a master server at 127.17.96.254:34043
I20260619 02:29:24.063742 18172 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:24.064288 18172 heartbeater.cc:511] Master 127.17.96.254:34043 requested a full tablet report, sending...
I20260619 02:29:24.065423 17847 ts_manager.cc:194] Registered new tserver with Master: 80001c82d6e640a4899a76d368ffc28b (127.17.96.194:32889)
I20260619 02:29:24.066118 17847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:57233
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
I20260619 02:29:24.078842 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:34043
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:34123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:24.198354 18177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:24.198673 18177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:24.198721 18177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:24.203400 18177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:24.203485 18177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:24.203598 18177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:29:24.209681 18177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:34123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:34043
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:24.210958 18177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:24.212066 18177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:24.222487 18182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:24.222479 18185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:24.222576 18177 server_base.cc:1034] running on GCE node
W20260619 02:29:24.222520 18183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:24.223268 18177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:24.224007 18177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:24.225221 18177 hybrid_clock.cc:648] HybridClock initialized: now 1781836164225143 us; error 86 us; skew 500 ppm
Jun 19 02:29:24 dist-test-slave-4qw6 krb5kdc[17802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836164, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:24.228708 18177 init.cc:377] Logged in from keytab as kudu/127.17.96.195@KRBTEST.COM (short username kudu)
I20260619 02:29:24.229830 18177 webserver.cc:460] Webserver started at http://127.17.96.195:38625/ using document root <none> and password file <none>
I20260619 02:29:24.230464 18177 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:24.230540 18177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:24.230794 18177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:24.233028 18177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "e7f43314527744219fa04630a33fdabc"
format_stamp: "Formatted at 2026-06-19 02:29:24 on dist-test-slave-4qw6"
server_key: "5d6b23e49d7b6d2ca40e754c6b0b265e"
server_key_iv: "7777068a2f17c1de71310218de48d7b4"
server_key_version: "encryptionkey@0"
I20260619 02:29:24.233668 18177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "e7f43314527744219fa04630a33fdabc"
format_stamp: "Formatted at 2026-06-19 02:29:24 on dist-test-slave-4qw6"
server_key: "5d6b23e49d7b6d2ca40e754c6b0b265e"
server_key_iv: "7777068a2f17c1de71310218de48d7b4"
server_key_version: "encryptionkey@0"
I20260619 02:29:24.237680 18177 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 02:29:24.240370 18192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:24.241719 18177 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260619 02:29:24.241925 18177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "e7f43314527744219fa04630a33fdabc"
format_stamp: "Formatted at 2026-06-19 02:29:24 on dist-test-slave-4qw6"
server_key: "5d6b23e49d7b6d2ca40e754c6b0b265e"
server_key_iv: "7777068a2f17c1de71310218de48d7b4"
server_key_version: "encryptionkey@0"
I20260619 02:29:24.242058 18177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:24.282712 18177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:24.285885 18177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:24.286128 18177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:24.286778 18177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:24.287678 18177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:24.287753 18177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:24.287828 18177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:24.287878 18177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:24.297374 18177 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:42421
I20260619 02:29:24.297397 18305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:42421 every 8 connection(s)
I20260619 02:29:24.298373 18177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:29:24.306779 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 18177
I20260619 02:29:24.306931 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:29:24.307250 17795 external_mini_cluster.cc:1428] Setting key 774109ceb75147068e245f6641210c74
Jun 19 02:29:24 dist-test-slave-4qw6 krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836164, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:24.313371 17885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:24.300373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:54083 (local address 127.17.96.254:34043)
0619 02:29:24.300538 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:24.300542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:24.301482 (+ 940us) server_negotiation.cc:407] Connection header received
0619 02:29:24.302976 (+ 1494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:24.302980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:24.303059 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:24.303144 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:24.304374 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:24.304911 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:24.306154 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:24.306380 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:24.306466 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:24.309640 (+ 3174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:24.309644 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:24.309647 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:24.309678 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:24.311340 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:24.311918 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:24.311921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:24.311922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:24.312004 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:24.312321 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:24.312323 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:24.312324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:24.312527 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:24.312603 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:24.313129 (+ 526us) server_negotiation.cc:299] Negotiation successful
0619 02:29:24.313240 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 02:29:24.314406 18308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:24.300803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:34043 (local address 127.17.96.195:54083)
0619 02:29:24.301321 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:24.301367 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:24.302718 (+ 1351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:24.303316 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:24.303323 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:24.303711 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:24.304232 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:24.304246 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:24.305030 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:24.305033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:24.306040 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:24.306051 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:24.306520 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:24.306523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:24.306667 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:24.307379 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:24.307402 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:24.309468 (+ 2066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:24.311483 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:24.311489 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:24.311504 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:24.311821 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:24.312088 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:24.312090 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:24.312092 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:24.312240 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:24.312618 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:24.312623 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:24.312900 (+ 277us) client_negotiation.cc:769] Sending connection context
0619 02:29:24.313083 (+ 183us) client_negotiation.cc:240] Negotiation successful
0619 02:29:24.313377 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":125,"threads_started":1}
I20260619 02:29:24.315701 18306 heartbeater.cc:347] Connected to a master server at 127.17.96.254:34043
I20260619 02:29:24.315984 18306 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:24.316541 18306 heartbeater.cc:511] Master 127.17.96.254:34043 requested a full tablet report, sending...
I20260619 02:29:24.317667 17847 ts_manager.cc:194] Registered new tserver with Master: e7f43314527744219fa04630a33fdabc (127.17.96.195:42421)
I20260619 02:29:24.318327 17847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:54083
I20260619 02:29:24.322469 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 02:29:24 dist-test-slave-4qw6 krb5kdc[17802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836164, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 02:29:24 dist-test-slave-4qw6 krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836164, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.96.193@KRBTEST.COM
I20260619 02:29:24.350258 18313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:24.339666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47108 (local address 127.17.96.193:42769)
0619 02:29:24.340033 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:24.340040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:24.340066 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:29:24.340211 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:24.340216 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:24.340427 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:24.340566 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:24.341282 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:24.342166 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:24.343033 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:24.343424 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:24.343515 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:24.346460 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:24.346472 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:24.346487 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:24.346518 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:24.348320 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:24.348811 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:24.348816 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:24.348818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:24.348900 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:24.349140 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:24.349142 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:24.349143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:24.349415 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:24.349587 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:24.349861 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 02:29:24.350032 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":146,"threads_started":1}
W20260619 02:29:24.351413 17952 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47108
Jun 19 02:29:24 dist-test-slave-4qw6 krb5kdc[17802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:29:24 dist-test-slave-4qw6 krb5kdc[17802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.193@KRBTEST.COM
I20260619 02:29:24.378885 18313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:24.370180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47122 (local address 127.17.96.193:42769)
0619 02:29:24.370368 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:24.370374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:24.370563 (+ 189us) server_negotiation.cc:407] Connection header received
0619 02:29:24.370756 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:24.370759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:24.370804 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:24.370897 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:24.371645 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:24.372164 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:24.373041 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:24.373293 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:24.373411 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:24.376087 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:24.376091 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:24.376093 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:24.376120 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:24.377363 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:24.377820 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:24.377823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:24.377826 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:24.377909 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:24.378153 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:24.378155 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:24.378156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:24.378328 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:24.378434 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:24.378649 (+ 215us) server_negotiation.cc:299] Negotiation successful
0619 02:29:24.378756 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 02:29:24.380119 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 17909
I20260619 02:29:24.386591 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 18043
I20260619 02:29:24.392764 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 18177
I20260619 02:29:24.398789 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 17815
2026-06-19T02:29:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2386 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:29:24 dist-test-slave-4qw6 krb5kdc[18320](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:29:24 dist-test-slave-4qw6 krb5kdc[18320](info): set up 2 sockets
Jun 19 02:29:24 dist-test-slave-4qw6 krb5kdc[18320](info): commencing operation
krb5kdc: starting...
W20260619 02:29:25.474000 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:29:25 dist-test-slave-4qw6 krb5kdc[18320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:29:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:29:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:29:25.603058 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:44835
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:42149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:44835
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:25.720335 18333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:25.720732 18333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:25.720813 18333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:25.725263 18333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:29:25.725402 18333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:25.725441 18333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:25.725486 18333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:29:25.725557 18333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:29:25.731601 18333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:42149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:44835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:44835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:25.732929 18333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:25.734144 18333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:25.743782 18341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:25.743808 18333 server_base.cc:1034] running on GCE node
W20260619 02:29:25.743793 18338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:25.743793 18339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:25.744684 18333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:25.746064 18333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:25.747277 18333 hybrid_clock.cc:648] HybridClock initialized: now 1781836165747242 us; error 57 us; skew 500 ppm
Jun 19 02:29:25 dist-test-slave-4qw6 krb5kdc[18320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836165, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:25.750501 18333 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:29:25.751499 18333 webserver.cc:460] Webserver started at http://127.17.96.254:42495/ using document root <none> and password file <none>
I20260619 02:29:25.752104 18333 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:25.752175 18333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:25.752362 18333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:25.754757 18333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "e7ca18c692d84c0a9739bf80f1e5456d"
format_stamp: "Formatted at 2026-06-19 02:29:25 on dist-test-slave-4qw6"
server_key: "da2a41ba722867472dd66faa67190566"
server_key_iv: "a3c2b388e1a3c04d625d43ec82171d56"
server_key_version: "encryptionkey@0"
I20260619 02:29:25.755353 18333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "e7ca18c692d84c0a9739bf80f1e5456d"
format_stamp: "Formatted at 2026-06-19 02:29:25 on dist-test-slave-4qw6"
server_key: "da2a41ba722867472dd66faa67190566"
server_key_iv: "a3c2b388e1a3c04d625d43ec82171d56"
server_key_version: "encryptionkey@0"
I20260619 02:29:25.759510 18333 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:29:25.762276 18348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:25.763736 18333 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:29:25.763928 18333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "e7ca18c692d84c0a9739bf80f1e5456d"
format_stamp: "Formatted at 2026-06-19 02:29:25 on dist-test-slave-4qw6"
server_key: "da2a41ba722867472dd66faa67190566"
server_key_iv: "a3c2b388e1a3c04d625d43ec82171d56"
server_key_version: "encryptionkey@0"
I20260619 02:29:25.764067 18333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:25.776925 18333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:25.780650 18333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:25.780894 18333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:25.788836 18333 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:44835
I20260619 02:29:25.788864 18400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:44835 every 8 connection(s)
I20260619 02:29:25.789965 18333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:29:25.790181 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 18333
I20260619 02:29:25.790300 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:29:25.790644 17795 external_mini_cluster.cc:1428] Setting key f0006b9058024d6d07fc45804d332f4c
I20260619 02:29:25.793644 18401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:25.799741 18401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7ca18c692d84c0a9739bf80f1e5456d: Bootstrap starting.
Jun 19 02:29:25 dist-test-slave-4qw6 krb5kdc[18320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:25.802284 18401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7ca18c692d84c0a9739bf80f1e5456d: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:25.803366 18401 log.cc:824] T 00000000000000000000000000000000 P e7ca18c692d84c0a9739bf80f1e5456d: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:25.805678 18404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:25.792169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40986 (local address 127.17.96.254:44835)
0619 02:29:25.792651 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:25.792668 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:25.792708 (+ 40us) server_negotiation.cc:407] Connection header received
0619 02:29:25.793391 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:25.793408 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:25.793862 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:25.794250 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:25.795122 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:25.796180 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:25.797094 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:25.797596 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:25.797803 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:25.800442 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:25.800448 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:25.800463 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:25.800495 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:25.802402 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:25.803014 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:25.803019 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:25.803024 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:25.803146 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:25.803468 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:25.803493 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:25.803497 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:25.804042 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:25.804255 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:25.804549 (+ 294us) server_negotiation.cc:299] Negotiation successful
0619 02:29:25.804896 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":160,"threads_started":1}
I20260619 02:29:25.805821 18401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7ca18c692d84c0a9739bf80f1e5456d: No bootstrap required, opened a new log
I20260619 02:29:25.810067 18401 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7ca18c692d84c0a9739bf80f1e5456d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7ca18c692d84c0a9739bf80f1e5456d" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44835 } }
I20260619 02:29:25.810348 18401 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7ca18c692d84c0a9739bf80f1e5456d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:25.810443 18401 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7ca18c692d84c0a9739bf80f1e5456d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ca18c692d84c0a9739bf80f1e5456d, State: Initialized, Role: FOLLOWER
I20260619 02:29:25.810946 18401 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7ca18c692d84c0a9739bf80f1e5456d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7ca18c692d84c0a9739bf80f1e5456d" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44835 } }
I20260619 02:29:25.811086 18401 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7ca18c692d84c0a9739bf80f1e5456d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:29:25.811179 18401 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7ca18c692d84c0a9739bf80f1e5456d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:29:25.811291 18401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7ca18c692d84c0a9739bf80f1e5456d [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:25.812381 18401 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7ca18c692d84c0a9739bf80f1e5456d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7ca18c692d84c0a9739bf80f1e5456d" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44835 } }
I20260619 02:29:25.812769 18401 leader_election.cc:302] T 00000000000000000000000000000000 P e7ca18c692d84c0a9739bf80f1e5456d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7ca18c692d84c0a9739bf80f1e5456d; no voters:
I20260619 02:29:25.813133 18401 leader_election.cc:288] T 00000000000000000000000000000000 P e7ca18c692d84c0a9739bf80f1e5456d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:29:25.813357 18406 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7ca18c692d84c0a9739bf80f1e5456d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:25.813705 18406 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7ca18c692d84c0a9739bf80f1e5456d [term 1 LEADER]: Becoming Leader. State: Replica: e7ca18c692d84c0a9739bf80f1e5456d, State: Running, Role: LEADER
I20260619 02:29:25.814080 18406 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7ca18c692d84c0a9739bf80f1e5456d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7ca18c692d84c0a9739bf80f1e5456d" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44835 } }
I20260619 02:29:25.814567 18401 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7ca18c692d84c0a9739bf80f1e5456d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:29:25.816004 18407 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7ca18c692d84c0a9739bf80f1e5456d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7ca18c692d84c0a9739bf80f1e5456d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7ca18c692d84c0a9739bf80f1e5456d" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44835 } } }
I20260619 02:29:25.816017 18408 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7ca18c692d84c0a9739bf80f1e5456d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7ca18c692d84c0a9739bf80f1e5456d. Latest consensus state: current_term: 1 leader_uuid: "e7ca18c692d84c0a9739bf80f1e5456d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7ca18c692d84c0a9739bf80f1e5456d" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44835 } } }
I20260619 02:29:25.816145 18407 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7ca18c692d84c0a9739bf80f1e5456d [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:25.816152 18408 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7ca18c692d84c0a9739bf80f1e5456d [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:25.816550 18416 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:29:25.820261 18416 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:29:25.827215 18416 catalog_manager.cc:1347] Generated new cluster ID: 3a49cb3a1bad4d329397fedfd749dd86
I20260619 02:29:25.827329 18416 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:29:25.869848 18416 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:29:25.870983 18416 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:29:25.878522 18416 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7ca18c692d84c0a9739bf80f1e5456d: Generated new TSK 0
I20260619 02:29:25.879364 18416 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.96.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
I20260619 02:29:25.917259 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:44835
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:42149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:26.044415 18427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:26.044729 18427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:26.044780 18427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:26.049430 18427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:26.049520 18427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:26.049589 18427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:29:26.055867 18427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:42149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:44835
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:26.057082 18427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:26.058259 18427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:26.069125 18435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:26.069238 18433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:26.069132 18432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:26.069406 18427 server_base.cc:1034] running on GCE node
I20260619 02:29:26.070156 18427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:26.070928 18427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:26.072142 18427 hybrid_clock.cc:648] HybridClock initialized: now 1781836166072105 us; error 65 us; skew 500 ppm
Jun 19 02:29:26 dist-test-slave-4qw6 krb5kdc[18320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836166, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:26.075381 18427 init.cc:377] Logged in from keytab as kudu/127.17.96.193@KRBTEST.COM (short username kudu)
I20260619 02:29:26.076342 18427 webserver.cc:460] Webserver started at http://127.17.96.193:37205/ using document root <none> and password file <none>
I20260619 02:29:26.076949 18427 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:26.077013 18427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:26.077194 18427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:26.079505 18427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "32a8f05db8574f97aa51b5fec6042d57"
format_stamp: "Formatted at 2026-06-19 02:29:26 on dist-test-slave-4qw6"
server_key: "ce4e377a1374b9cfb21a393d07b3644e"
server_key_iv: "0a7ba40305f85397ab53695ba1897b66"
server_key_version: "encryptionkey@0"
I20260619 02:29:26.080070 18427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "32a8f05db8574f97aa51b5fec6042d57"
format_stamp: "Formatted at 2026-06-19 02:29:26 on dist-test-slave-4qw6"
server_key: "ce4e377a1374b9cfb21a393d07b3644e"
server_key_iv: "0a7ba40305f85397ab53695ba1897b66"
server_key_version: "encryptionkey@0"
I20260619 02:29:26.084221 18427 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260619 02:29:26.087102 18442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:26.088892 18427 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:29:26.089035 18427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "32a8f05db8574f97aa51b5fec6042d57"
format_stamp: "Formatted at 2026-06-19 02:29:26 on dist-test-slave-4qw6"
server_key: "ce4e377a1374b9cfb21a393d07b3644e"
server_key_iv: "0a7ba40305f85397ab53695ba1897b66"
server_key_version: "encryptionkey@0"
I20260619 02:29:26.089166 18427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:26.118286 18427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:26.121923 18427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:26.122098 18427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:26.122679 18427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:26.123553 18427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:26.123603 18427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:26.123646 18427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:26.123664 18427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:26.133881 18427 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:37207
I20260619 02:29:26.133910 18555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:37207 every 8 connection(s)
I20260619 02:29:26.135057 18427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
Jun 19 02:29:26 dist-test-slave-4qw6 krb5kdc[18320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836166, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:26.145398 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 18427
I20260619 02:29:26.145502 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:29:26.145889 17795 external_mini_cluster.cc:1428] Setting key e4641d50395e93e5983013172d994e64
I20260619 02:29:26.150720 18404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:26.137229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:60431 (local address 127.17.96.254:44835)
0619 02:29:26.137411 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:26.137416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:26.138434 (+ 1018us) server_negotiation.cc:407] Connection header received
0619 02:29:26.139440 (+ 1006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:26.139444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:26.139509 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:26.139631 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:26.140984 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.141551 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.142855 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.143111 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.143238 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:26.146299 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:26.146302 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:26.146304 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:26.146334 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:26.148233 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:26.148827 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:26.148834 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:26.148839 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:26.148962 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:26.149348 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:26.149351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:26.149352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:26.149560 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:26.149679 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:26.150365 (+ 686us) server_negotiation.cc:299] Negotiation successful
0619 02:29:26.150529 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 02:29:26.151662 18558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:26.137660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:44835 (local address 127.17.96.193:60431)
0619 02:29:26.138248 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:26.138298 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:26.139204 (+ 906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:26.139824 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:26.139837 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:26.140253 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:26.140788 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:26.140803 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.141699 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.141702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:26.142706 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:26.142717 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.143285 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.143289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:26.143477 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:26.144103 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:26.144121 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:26.146086 (+ 1965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:26.148384 (+ 2298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:26.148390 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:26.148406 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:26.148706 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:26.149059 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:26.149061 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:26.149062 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:26.149208 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:26.149690 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:26.149699 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:26.150079 (+ 380us) client_negotiation.cc:769] Sending connection context
0619 02:29:26.150296 (+ 217us) client_negotiation.cc:240] Negotiation successful
0619 02:29:26.150686 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":154,"threads_started":1}
I20260619 02:29:26.153182 18556 heartbeater.cc:347] Connected to a master server at 127.17.96.254:44835
I20260619 02:29:26.153501 18556 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:26.154139 18556 heartbeater.cc:511] Master 127.17.96.254:44835 requested a full tablet report, sending...
I20260619 02:29:26.156198 18365 ts_manager.cc:194] Registered new tserver with Master: 32a8f05db8574f97aa51b5fec6042d57 (127.17.96.193:37207)
WARNING: no policy specified for kudu/127.17.96.194@KRBTEST.COM; defaulting to no policy
I20260619 02:29:26.157744 18365 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:60431
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
I20260619 02:29:26.177224 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:44835
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:42149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:26.308526 18561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:26.308864 18561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:26.308913 18561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:26.313175 18561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:26.313268 18561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:26.313340 18561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:29:26.319669 18561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:42149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:44835
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:26.320863 18561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:26.322167 18561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:29:26.334101 18561 server_base.cc:1034] running on GCE node
W20260619 02:29:26.334036 18566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:26.334045 18569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:26.334039 18567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:26.334746 18561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:26.335503 18561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:26.336700 18561 hybrid_clock.cc:648] HybridClock initialized: now 1781836166336667 us; error 53 us; skew 500 ppm
Jun 19 02:29:26 dist-test-slave-4qw6 krb5kdc[18320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836166, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:26.339728 18561 init.cc:377] Logged in from keytab as kudu/127.17.96.194@KRBTEST.COM (short username kudu)
I20260619 02:29:26.340776 18561 webserver.cc:460] Webserver started at http://127.17.96.194:44295/ using document root <none> and password file <none>
I20260619 02:29:26.341367 18561 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:26.341431 18561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:26.341620 18561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:26.343910 18561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "c95d1753f4d2494696134db00f4504e6"
format_stamp: "Formatted at 2026-06-19 02:29:26 on dist-test-slave-4qw6"
server_key: "f6ce8c8c08958cfe8bd429ba795a0587"
server_key_iv: "3b5be51139004dcc74335ab15e3000f1"
server_key_version: "encryptionkey@0"
I20260619 02:29:26.344514 18561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "c95d1753f4d2494696134db00f4504e6"
format_stamp: "Formatted at 2026-06-19 02:29:26 on dist-test-slave-4qw6"
server_key: "f6ce8c8c08958cfe8bd429ba795a0587"
server_key_iv: "3b5be51139004dcc74335ab15e3000f1"
server_key_version: "encryptionkey@0"
I20260619 02:29:26.348649 18561 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 02:29:26.351334 18576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:26.352680 18561 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 02:29:26.352847 18561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "c95d1753f4d2494696134db00f4504e6"
format_stamp: "Formatted at 2026-06-19 02:29:26 on dist-test-slave-4qw6"
server_key: "f6ce8c8c08958cfe8bd429ba795a0587"
server_key_iv: "3b5be51139004dcc74335ab15e3000f1"
server_key_version: "encryptionkey@0"
I20260619 02:29:26.352982 18561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:26.370517 18561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:26.374177 18561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:26.374423 18561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:26.375064 18561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:26.375977 18561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:26.376055 18561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:26.376148 18561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:26.376199 18561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:26.386082 18561 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:43059
I20260619 02:29:26.386127 18689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:43059 every 8 connection(s)
I20260619 02:29:26.387166 18561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:29:26.395687 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 18561
I20260619 02:29:26.395814 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:29:26.396087 17795 external_mini_cluster.cc:1428] Setting key dce4a6a622bfa6d4a1fe039053702fad
Jun 19 02:29:26 dist-test-slave-4qw6 krb5kdc[18320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836166, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:26.401665 18404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:26.389202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:58279 (local address 127.17.96.254:44835)
0619 02:29:26.389345 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:26.389348 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:26.390360 (+ 1012us) server_negotiation.cc:407] Connection header received
0619 02:29:26.391252 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:26.391255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:26.391306 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:26.391405 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:26.392731 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.393278 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.394762 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.395000 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.395088 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:26.397896 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:26.397899 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:26.397901 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:26.397926 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:26.399408 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:26.399999 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:26.400003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:26.400006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:26.400091 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:26.400453 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:26.400456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:26.400458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:26.400664 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:26.400776 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:26.401334 (+ 558us) server_negotiation.cc:299] Negotiation successful
0619 02:29:26.401488 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 02:29:26.402570 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:26.389621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:44835 (local address 127.17.96.194:58279)
0619 02:29:26.390188 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:26.390235 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:26.391042 (+ 807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:26.391576 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:26.391583 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:26.391985 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:26.392567 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:26.392585 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.393429 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.393432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:26.394637 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:26.394649 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.395146 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.395149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:26.395308 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:26.396014 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:26.396031 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:26.397686 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:26.399562 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:26.399569 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:26.399588 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:26.399880 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:26.400189 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:26.400192 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:26.400194 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:26.400348 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:26.400775 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:26.400783 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:26.401066 (+ 283us) client_negotiation.cc:769] Sending connection context
0619 02:29:26.401279 (+ 213us) client_negotiation.cc:240] Negotiation successful
0619 02:29:26.401646 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":197,"threads_started":1}
I20260619 02:29:26.403870 18690 heartbeater.cc:347] Connected to a master server at 127.17.96.254:44835
I20260619 02:29:26.404153 18690 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:26.404703 18690 heartbeater.cc:511] Master 127.17.96.254:44835 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.96.195@KRBTEST.COM; defaulting to no policy
I20260619 02:29:26.405987 18365 ts_manager.cc:194] Registered new tserver with Master: c95d1753f4d2494696134db00f4504e6 (127.17.96.194:43059)
I20260619 02:29:26.406648 18365 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:58279
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
I20260619 02:29:26.424698 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:44835
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:42149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:26.545885 18695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:26.546206 18695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:26.546257 18695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:26.550815 18695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:26.550895 18695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:26.550964 18695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:29:26.556900 18695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:42149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:44835
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:26.558228 18695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:26.559298 18695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:26.569883 18701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:26.569882 18700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:26.569887 18703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:26.570034 18695 server_base.cc:1034] running on GCE node
I20260619 02:29:26.570703 18695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:26.571506 18695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:26.572719 18695 hybrid_clock.cc:648] HybridClock initialized: now 1781836166572696 us; error 38 us; skew 500 ppm
Jun 19 02:29:26 dist-test-slave-4qw6 krb5kdc[18320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836166, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:26.576045 18695 init.cc:377] Logged in from keytab as kudu/127.17.96.195@KRBTEST.COM (short username kudu)
I20260619 02:29:26.577172 18695 webserver.cc:460] Webserver started at http://127.17.96.195:43797/ using document root <none> and password file <none>
I20260619 02:29:26.577910 18695 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:26.578024 18695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:26.578305 18695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:26.580665 18695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "d7cb594b1cf54c678f7768c762c0419f"
format_stamp: "Formatted at 2026-06-19 02:29:26 on dist-test-slave-4qw6"
server_key: "cfb116e884dfa6885b735cc5f6f24caa"
server_key_iv: "8ef9fb5e0260ade447c74ea6050cd0eb"
server_key_version: "encryptionkey@0"
I20260619 02:29:26.581355 18695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "d7cb594b1cf54c678f7768c762c0419f"
format_stamp: "Formatted at 2026-06-19 02:29:26 on dist-test-slave-4qw6"
server_key: "cfb116e884dfa6885b735cc5f6f24caa"
server_key_iv: "8ef9fb5e0260ade447c74ea6050cd0eb"
server_key_version: "encryptionkey@0"
I20260619 02:29:26.585907 18695 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260619 02:29:26.588985 18710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:26.590481 18695 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:29:26.590662 18695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "d7cb594b1cf54c678f7768c762c0419f"
format_stamp: "Formatted at 2026-06-19 02:29:26 on dist-test-slave-4qw6"
server_key: "cfb116e884dfa6885b735cc5f6f24caa"
server_key_iv: "8ef9fb5e0260ade447c74ea6050cd0eb"
server_key_version: "encryptionkey@0"
I20260619 02:29:26.590821 18695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:26.609390 18695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:26.612955 18695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:26.613214 18695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:26.614038 18695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:26.615140 18695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:26.615231 18695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:26.615300 18695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:26.615330 18695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:26.626202 18695 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:40737
I20260619 02:29:26.626233 18823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:40737 every 8 connection(s)
I20260619 02:29:26.627210 18695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:29:26.632215 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 18695
I20260619 02:29:26.632364 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:29:26.632651 17795 external_mini_cluster.cc:1428] Setting key e59b3cc2aef58ca2715976efdcd86680
Jun 19 02:29:26 dist-test-slave-4qw6 krb5kdc[18320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836166, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:26.641831 18404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:26.629264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:48113 (local address 127.17.96.254:44835)
0619 02:29:26.629423 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:26.629427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:26.630399 (+ 972us) server_negotiation.cc:407] Connection header received
0619 02:29:26.631437 (+ 1038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:26.631440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:26.631492 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:26.631573 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:26.633027 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.633755 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.635365 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.635586 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.635670 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:26.638374 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:26.638377 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:26.638380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:26.638408 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:26.639883 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:26.640398 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:26.640400 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:26.640402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:26.640483 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:26.640773 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:26.640776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:26.640777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:26.640987 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:26.641061 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:26.641542 (+ 481us) server_negotiation.cc:299] Negotiation successful
0619 02:29:26.641650 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 02:29:26.642786 18826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:26.629689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:44835 (local address 127.17.96.195:48113)
0619 02:29:26.630210 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:26.630259 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:26.631217 (+ 958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:26.631797 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:26.631804 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:26.632200 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:26.632841 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:26.632861 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.633921 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.633927 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:26.635241 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:26.635253 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.635720 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.635723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:26.635886 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:26.636489 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:26.636506 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:26.638182 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:26.640003 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:26.640008 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:26.640022 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:26.640306 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:26.640563 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:26.640565 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:26.640567 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:26.640698 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:26.641072 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:26.641077 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:26.641333 (+ 256us) client_negotiation.cc:769] Sending connection context
0619 02:29:26.641499 (+ 166us) client_negotiation.cc:240] Negotiation successful
0619 02:29:26.641855 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":130,"threads_started":1}
I20260619 02:29:26.644086 18824 heartbeater.cc:347] Connected to a master server at 127.17.96.254:44835
I20260619 02:29:26.644369 18824 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:26.644935 18824 heartbeater.cc:511] Master 127.17.96.254:44835 requested a full tablet report, sending...
I20260619 02:29:26.646083 18363 ts_manager.cc:194] Registered new tserver with Master: d7cb594b1cf54c678f7768c762c0419f (127.17.96.195:40737)
I20260619 02:29:26.646799 18363 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:48113
I20260619 02:29:26.646942 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:29:26.658378 18404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:26.650566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40996 (local address 127.17.96.254:44835)
0619 02:29:26.650723 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:26.650730 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:26.651024 (+ 294us) server_negotiation.cc:407] Connection header received
0619 02:29:26.651171 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:26.651173 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:26.651217 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:26.651279 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:26.652182 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.652712 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.653539 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.653740 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.653887 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:26.655323 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:26.655326 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:26.655328 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:26.655346 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:26.656639 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:26.657106 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:26.657109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:26.657110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:26.657187 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:26.657424 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:26.657426 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:26.657427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:26.657645 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:26.657818 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:26.658072 (+ 254us) server_negotiation.cc:299] Negotiation successful
0619 02:29:26.658177 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 02:29:26.663760 18363 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40996:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:29:26.666960 18363 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:29:26.683184 18839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:26.677535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49606 (local address 127.17.96.193:37207)
0619 02:29:26.677967 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:26.677973 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:26.678500 (+ 527us) server_negotiation.cc:407] Connection header received
0619 02:29:26.678610 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:26.678618 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:26.678800 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:26.678937 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:26.679194 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.679998 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.681621 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.682339 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.682492 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:26.682673 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:26.682863 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 02:29:26.682984 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":95,"threads_started":1}
I20260619 02:29:26.683174 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:26.677376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:37207 (local address 127.0.0.1:49606)
0619 02:29:26.678378 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:26.678393 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:26.678501 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:26.678917 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:26.678920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:26.678939 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:26.679065 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:26.679075 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.680170 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.680174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:26.681475 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:26.681486 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.682486 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.682490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:26.682608 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:26.682647 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 02:29:26.682825 (+ 178us) client_negotiation.cc:240] Negotiation successful
0619 02:29:26.682987 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":943,"mutex_wait_us":669,"thread_start_us":110,"threads_started":1}
I20260619 02:29:26.684384 18837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:26.676639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:43059 (local address 127.0.0.1:60282)
0619 02:29:26.678078 (+ 1439us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:26.678090 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:26.678216 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:26.678733 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:26.678741 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:26.678766 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:26.679066 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:26.679075 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.680635 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.680638 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:26.682374 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:26.682386 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.683798 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.683802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:26.683989 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:26.684029 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 02:29:26.684104 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 02:29:26.684226 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1350,"mutex_wait_us":336,"thread_start_us":109,"threads_started":1}
I20260619 02:29:26.684543 18836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:26.676557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47098 (local address 127.17.96.195:40737)
0619 02:29:26.677693 (+ 1136us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:26.677712 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:26.677740 (+ 28us) server_negotiation.cc:407] Connection header received
0619 02:29:26.678097 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:26.678106 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:26.678379 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:26.678545 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:26.679213 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.680460 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.682538 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.683474 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.683588 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:26.683727 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:26.684262 (+ 535us) server_negotiation.cc:299] Negotiation successful
0619 02:29:26.684371 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1033,"thread_start_us":143,"threads_started":1}
I20260619 02:29:26.684875 18835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:26.676426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:40737 (local address 127.0.0.1:47098)
0619 02:29:26.677585 (+ 1159us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:26.677648 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:26.677832 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:26.678760 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:26.678762 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:26.678785 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:26.679066 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:26.679075 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.680635 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.680638 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:26.682374 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:26.682386 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.683801 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.683804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:26.683986 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:26.684034 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 02:29:26.684104 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 02:29:26.684782 (+ 678us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":964,"mutex_wait_us":900,"thread_start_us":94,"threads_started":1}
I20260619 02:29:26.686118 18838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:26.676790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60282 (local address 127.17.96.194:43059)
0619 02:29:26.677522 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:26.677536 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:26.678190 (+ 654us) server_negotiation.cc:407] Connection header received
0619 02:29:26.678339 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:26.678343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:26.678553 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:26.678714 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:26.679214 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.680463 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.682539 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:26.683474 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:26.683588 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:26.683727 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:26.685849 (+ 2122us) server_negotiation.cc:299] Negotiation successful
0619 02:29:26.685970 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":90,"threads_started":1}
I20260619 02:29:26.687062 18490 tablet_service.cc:1467] Processing CreateTablet for tablet 075bc2aa8bad45298a553c8a315188a7 (DEFAULT_TABLE table=test-table [id=ba99bfed49434ef2be8c262e825f687f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:26.687729 18758 tablet_service.cc:1467] Processing CreateTablet for tablet 075bc2aa8bad45298a553c8a315188a7 (DEFAULT_TABLE table=test-table [id=ba99bfed49434ef2be8c262e825f687f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:26.688323 18490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 075bc2aa8bad45298a553c8a315188a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:26.688874 18758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 075bc2aa8bad45298a553c8a315188a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:26.689159 18624 tablet_service.cc:1467] Processing CreateTablet for tablet 075bc2aa8bad45298a553c8a315188a7 (DEFAULT_TABLE table=test-table [id=ba99bfed49434ef2be8c262e825f687f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:26.690287 18624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 075bc2aa8bad45298a553c8a315188a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:26.695291 18841 tablet_bootstrap.cc:492] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57: Bootstrap starting.
I20260619 02:29:26.696787 18842 tablet_bootstrap.cc:492] T 075bc2aa8bad45298a553c8a315188a7 P d7cb594b1cf54c678f7768c762c0419f: Bootstrap starting.
I20260619 02:29:26.698079 18843 tablet_bootstrap.cc:492] T 075bc2aa8bad45298a553c8a315188a7 P c95d1753f4d2494696134db00f4504e6: Bootstrap starting.
I20260619 02:29:26.698315 18841 tablet_bootstrap.cc:654] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:26.699368 18842 tablet_bootstrap.cc:654] T 075bc2aa8bad45298a553c8a315188a7 P d7cb594b1cf54c678f7768c762c0419f: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:26.699900 18841 log.cc:824] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:26.700683 18842 log.cc:824] T 075bc2aa8bad45298a553c8a315188a7 P d7cb594b1cf54c678f7768c762c0419f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:26.700888 18843 tablet_bootstrap.cc:654] T 075bc2aa8bad45298a553c8a315188a7 P c95d1753f4d2494696134db00f4504e6: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:26.701735 18843 log.cc:824] T 075bc2aa8bad45298a553c8a315188a7 P c95d1753f4d2494696134db00f4504e6: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:26.702363 18841 tablet_bootstrap.cc:492] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57: No bootstrap required, opened a new log
I20260619 02:29:26.702608 18841 ts_tablet_manager.cc:1397] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260619 02:29:26.702872 18842 tablet_bootstrap.cc:492] T 075bc2aa8bad45298a553c8a315188a7 P d7cb594b1cf54c678f7768c762c0419f: No bootstrap required, opened a new log
I20260619 02:29:26.703078 18842 ts_tablet_manager.cc:1397] T 075bc2aa8bad45298a553c8a315188a7 P d7cb594b1cf54c678f7768c762c0419f: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260619 02:29:26.703688 18843 tablet_bootstrap.cc:492] T 075bc2aa8bad45298a553c8a315188a7 P c95d1753f4d2494696134db00f4504e6: No bootstrap required, opened a new log
I20260619 02:29:26.703908 18843 ts_tablet_manager.cc:1397] T 075bc2aa8bad45298a553c8a315188a7 P c95d1753f4d2494696134db00f4504e6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 02:29:26.707250 18843 raft_consensus.cc:348] T 075bc2aa8bad45298a553c8a315188a7 P c95d1753f4d2494696134db00f4504e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7cb594b1cf54c678f7768c762c0419f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 40737 } } peers { permanent_uuid: "c95d1753f4d2494696134db00f4504e6" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43059 } } peers { permanent_uuid: "32a8f05db8574f97aa51b5fec6042d57" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37207 } }
I20260619 02:29:26.707250 18842 raft_consensus.cc:348] T 075bc2aa8bad45298a553c8a315188a7 P d7cb594b1cf54c678f7768c762c0419f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7cb594b1cf54c678f7768c762c0419f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 40737 } } peers { permanent_uuid: "c95d1753f4d2494696134db00f4504e6" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43059 } } peers { permanent_uuid: "32a8f05db8574f97aa51b5fec6042d57" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37207 } }
I20260619 02:29:26.707298 18841 raft_consensus.cc:348] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7cb594b1cf54c678f7768c762c0419f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 40737 } } peers { permanent_uuid: "c95d1753f4d2494696134db00f4504e6" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43059 } } peers { permanent_uuid: "32a8f05db8574f97aa51b5fec6042d57" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37207 } }
I20260619 02:29:26.707579 18842 raft_consensus.cc:374] T 075bc2aa8bad45298a553c8a315188a7 P d7cb594b1cf54c678f7768c762c0419f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:26.707579 18841 raft_consensus.cc:374] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:26.707579 18843 raft_consensus.cc:374] T 075bc2aa8bad45298a553c8a315188a7 P c95d1753f4d2494696134db00f4504e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:26.707656 18843 raft_consensus.cc:729] T 075bc2aa8bad45298a553c8a315188a7 P c95d1753f4d2494696134db00f4504e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95d1753f4d2494696134db00f4504e6, State: Initialized, Role: FOLLOWER
I20260619 02:29:26.707647 18841 raft_consensus.cc:729] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32a8f05db8574f97aa51b5fec6042d57, State: Initialized, Role: FOLLOWER
I20260619 02:29:26.707691 18842 raft_consensus.cc:729] T 075bc2aa8bad45298a553c8a315188a7 P d7cb594b1cf54c678f7768c762c0419f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7cb594b1cf54c678f7768c762c0419f, State: Initialized, Role: FOLLOWER
I20260619 02:29:26.708186 18843 consensus_queue.cc:260] T 075bc2aa8bad45298a553c8a315188a7 P c95d1753f4d2494696134db00f4504e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7cb594b1cf54c678f7768c762c0419f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 40737 } } peers { permanent_uuid: "c95d1753f4d2494696134db00f4504e6" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43059 } } peers { permanent_uuid: "32a8f05db8574f97aa51b5fec6042d57" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37207 } }
I20260619 02:29:26.708194 18841 consensus_queue.cc:260] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7cb594b1cf54c678f7768c762c0419f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 40737 } } peers { permanent_uuid: "c95d1753f4d2494696134db00f4504e6" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43059 } } peers { permanent_uuid: "32a8f05db8574f97aa51b5fec6042d57" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37207 } }
I20260619 02:29:26.708194 18842 consensus_queue.cc:260] T 075bc2aa8bad45298a553c8a315188a7 P d7cb594b1cf54c678f7768c762c0419f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7cb594b1cf54c678f7768c762c0419f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 40737 } } peers { permanent_uuid: "c95d1753f4d2494696134db00f4504e6" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43059 } } peers { permanent_uuid: "32a8f05db8574f97aa51b5fec6042d57" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37207 } }
I20260619 02:29:26.708986 18556 heartbeater.cc:503] Master 127.17.96.254:44835 was elected leader, sending a full tablet report...
I20260619 02:29:26.709388 18843 ts_tablet_manager.cc:1428] T 075bc2aa8bad45298a553c8a315188a7 P c95d1753f4d2494696134db00f4504e6: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 02:29:26.709462 18690 heartbeater.cc:503] Master 127.17.96.254:44835 was elected leader, sending a full tablet report...
I20260619 02:29:26.709503 18824 heartbeater.cc:503] Master 127.17.96.254:44835 was elected leader, sending a full tablet report...
I20260619 02:29:26.710500 18842 ts_tablet_manager.cc:1428] T 075bc2aa8bad45298a553c8a315188a7 P d7cb594b1cf54c678f7768c762c0419f: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260619 02:29:26.714512 18841 ts_tablet_manager.cc:1428] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
W20260619 02:29:26.878345 18825 tablet.cc:2369] T 075bc2aa8bad45298a553c8a315188a7 P d7cb594b1cf54c678f7768c762c0419f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:29:26.886652 18557 tablet.cc:2369] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:29:26.888396 18691 tablet.cc:2369] T 075bc2aa8bad45298a553c8a315188a7 P c95d1753f4d2494696134db00f4504e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:29:27.055377 18847 raft_consensus.cc:482] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:29:27.055653 18847 raft_consensus.cc:504] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7cb594b1cf54c678f7768c762c0419f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 40737 } } peers { permanent_uuid: "c95d1753f4d2494696134db00f4504e6" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43059 } } peers { permanent_uuid: "32a8f05db8574f97aa51b5fec6042d57" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37207 } }
I20260619 02:29:27.057003 18847 leader_election.cc:288] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7cb594b1cf54c678f7768c762c0419f (127.17.96.195:40737), c95d1753f4d2494696134db00f4504e6 (127.17.96.194:43059)
I20260619 02:29:27.061343 18838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:27.057575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:43405 (local address 127.17.96.194:43059)
0619 02:29:27.057729 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:27.057734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:27.057929 (+ 195us) server_negotiation.cc:407] Connection header received
0619 02:29:27.058067 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:27.058072 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:27.058126 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:27.058222 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:27.058576 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:27.059090 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:27.060295 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:27.060706 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:27.060846 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:27.060907 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:27.061109 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 02:29:27.061201 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 02:29:27.061386 18836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:27.057401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:55635 (local address 127.17.96.195:40737)
0619 02:29:27.057575 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:27.057581 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:27.057866 (+ 285us) server_negotiation.cc:407] Connection header received
0619 02:29:27.058067 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:27.058070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:27.058123 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:27.058183 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:27.058597 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:27.059095 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:27.060289 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:27.060719 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:27.060869 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:27.060924 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:27.061192 (+ 268us) server_negotiation.cc:299] Negotiation successful
0619 02:29:27.061276 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 02:29:27.061371 18851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:27.057352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:43059 (local address 127.17.96.193:43405)
0619 02:29:27.057796 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:27.057813 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:27.057954 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:27.058251 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:27.058254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:27.058270 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:27.058453 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:27.058460 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:27.059216 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:27.059221 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:27.060148 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:27.060158 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:27.060846 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:27.060850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:27.060954 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:27.061001 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 02:29:27.061062 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 02:29:27.061124 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":52,"thread_start_us":131,"threads_started":1}
I20260619 02:29:27.061391 18850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:27.057333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:40737 (local address 127.17.96.193:55635)
0619 02:29:27.057706 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:27.057720 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:27.057929 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:27.058228 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:27.058231 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:27.058246 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:27.058464 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:27.058470 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:27.059219 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:27.059224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:27.060155 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:27.060164 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:27.060871 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:27.060875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:27.061029 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:27.061077 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 02:29:27.061159 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 02:29:27.061227 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":126,"spinlock_wait_cycles":18304,"thread_start_us":104,"threads_started":1}
I20260619 02:29:27.062291 18644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "075bc2aa8bad45298a553c8a315188a7" candidate_uuid: "32a8f05db8574f97aa51b5fec6042d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95d1753f4d2494696134db00f4504e6" is_pre_election: true
I20260619 02:29:27.062379 18778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "075bc2aa8bad45298a553c8a315188a7" candidate_uuid: "32a8f05db8574f97aa51b5fec6042d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7cb594b1cf54c678f7768c762c0419f" is_pre_election: true
I20260619 02:29:27.062675 18644 raft_consensus.cc:2454] T 075bc2aa8bad45298a553c8a315188a7 P c95d1753f4d2494696134db00f4504e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32a8f05db8574f97aa51b5fec6042d57 in term 0.
I20260619 02:29:27.062760 18778 raft_consensus.cc:2454] T 075bc2aa8bad45298a553c8a315188a7 P d7cb594b1cf54c678f7768c762c0419f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32a8f05db8574f97aa51b5fec6042d57 in term 0.
I20260619 02:29:27.063304 18444 leader_election.cc:302] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 32a8f05db8574f97aa51b5fec6042d57, c95d1753f4d2494696134db00f4504e6, d7cb594b1cf54c678f7768c762c0419f; no voters:
I20260619 02:29:27.063673 18847 raft_consensus.cc:2781] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:29:27.063778 18847 raft_consensus.cc:482] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:29:27.063843 18847 raft_consensus.cc:3037] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:27.065101 18847 raft_consensus.cc:504] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7cb594b1cf54c678f7768c762c0419f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 40737 } } peers { permanent_uuid: "c95d1753f4d2494696134db00f4504e6" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43059 } } peers { permanent_uuid: "32a8f05db8574f97aa51b5fec6042d57" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37207 } }
I20260619 02:29:27.065608 18847 leader_election.cc:288] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57 [CANDIDATE]: Term 1 election: Requested vote from peers d7cb594b1cf54c678f7768c762c0419f (127.17.96.195:40737), c95d1753f4d2494696134db00f4504e6 (127.17.96.194:43059)
I20260619 02:29:27.066148 18778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "075bc2aa8bad45298a553c8a315188a7" candidate_uuid: "32a8f05db8574f97aa51b5fec6042d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7cb594b1cf54c678f7768c762c0419f"
I20260619 02:29:27.066167 18644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "075bc2aa8bad45298a553c8a315188a7" candidate_uuid: "32a8f05db8574f97aa51b5fec6042d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95d1753f4d2494696134db00f4504e6"
I20260619 02:29:27.066293 18778 raft_consensus.cc:3037] T 075bc2aa8bad45298a553c8a315188a7 P d7cb594b1cf54c678f7768c762c0419f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:27.066371 18644 raft_consensus.cc:3037] T 075bc2aa8bad45298a553c8a315188a7 P c95d1753f4d2494696134db00f4504e6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:27.067768 18778 raft_consensus.cc:2454] T 075bc2aa8bad45298a553c8a315188a7 P d7cb594b1cf54c678f7768c762c0419f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32a8f05db8574f97aa51b5fec6042d57 in term 1.
I20260619 02:29:27.067749 18644 raft_consensus.cc:2454] T 075bc2aa8bad45298a553c8a315188a7 P c95d1753f4d2494696134db00f4504e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32a8f05db8574f97aa51b5fec6042d57 in term 1.
I20260619 02:29:27.068173 18444 leader_election.cc:302] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32a8f05db8574f97aa51b5fec6042d57, d7cb594b1cf54c678f7768c762c0419f; no voters:
I20260619 02:29:27.068460 18847 raft_consensus.cc:2781] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:27.068764 18847 raft_consensus.cc:686] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57 [term 1 LEADER]: Becoming Leader. State: Replica: 32a8f05db8574f97aa51b5fec6042d57, State: Running, Role: LEADER
I20260619 02:29:27.069196 18847 consensus_queue.cc:237] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7cb594b1cf54c678f7768c762c0419f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 40737 } } peers { permanent_uuid: "c95d1753f4d2494696134db00f4504e6" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43059 } } peers { permanent_uuid: "32a8f05db8574f97aa51b5fec6042d57" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37207 } }
I20260619 02:29:27.073503 18365 catalog_manager.cc:5495] T 075bc2aa8bad45298a553c8a315188a7 P 32a8f05db8574f97aa51b5fec6042d57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32a8f05db8574f97aa51b5fec6042d57 (127.17.96.193). New cstate: current_term: 1 leader_uuid: "32a8f05db8574f97aa51b5fec6042d57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7cb594b1cf54c678f7768c762c0419f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 40737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95d1753f4d2494696134db00f4504e6" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32a8f05db8574f97aa51b5fec6042d57" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37207 } health_report { overall_health: HEALTHY } } }
Jun 19 02:29:27 dist-test-slave-4qw6 krb5kdc[18320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.193@KRBTEST.COM
I20260619 02:29:27.144078 18839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:27.129923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49620 (local address 127.17.96.193:37207)
0619 02:29:27.130125 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:27.130130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:27.130287 (+ 157us) server_negotiation.cc:407] Connection header received
0619 02:29:27.130488 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:27.130491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:27.130549 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:27.130643 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:27.131390 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:27.131941 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:27.132965 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:27.133221 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:27.133346 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:27.139700 (+ 6354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:27.139709 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:27.139724 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:27.139754 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:27.142009 (+ 2255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:27.142561 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:27.142568 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:27.142570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:27.142673 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:27.142972 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:27.142975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:27.142977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:27.143298 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:27.143518 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:27.143727 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 02:29:27.143867 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
Jun 19 02:29:27 dist-test-slave-4qw6 krb5kdc[18320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836167, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 02:29:27 dist-test-slave-4qw6 krb5kdc[18320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836167, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.96.193@KRBTEST.COM
I20260619 02:29:27.174853 18839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:27.164774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49624 (local address 127.17.96.193:37207)
0619 02:29:27.164945 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:27.164950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:27.165141 (+ 191us) server_negotiation.cc:407] Connection header received
0619 02:29:27.165384 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:27.165387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:27.165436 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:27.165542 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:27.166445 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:27.166983 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:27.168317 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:27.168568 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:27.168740 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:27.171557 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:27.171560 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:27.171562 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:27.171592 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:27.172905 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:27.173398 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:27.173403 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:27.173405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:27.173494 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:27.173857 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:27.173863 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:27.173864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:27.174058 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:27.174240 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:27.174541 (+ 301us) server_negotiation.cc:299] Negotiation successful
0619 02:29:27.174677 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Jun 19 02:29:27 dist-test-slave-4qw6 krb5kdc[18320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:29:27 dist-test-slave-4qw6 krb5kdc[18320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.193@KRBTEST.COM
I20260619 02:29:27.206101 18839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:27.195726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49636 (local address 127.17.96.193:37207)
0619 02:29:27.195995 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:27.196000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:27.196205 (+ 205us) server_negotiation.cc:407] Connection header received
0619 02:29:27.196388 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:27.196391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:27.196449 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:27.196547 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:27.197866 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:27.198420 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:27.199386 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:27.199620 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:27.199712 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:27.202765 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:27.202769 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:27.202771 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:27.202802 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:27.204197 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:27.204763 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:27.204766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:27.204767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:27.204850 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:27.205133 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:27.205135 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:27.205136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:27.205342 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:27.205452 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:27.205711 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 02:29:27.205925 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
W20260619 02:29:27.207710 18470 tablet_service.cc:2908] Rejecting scan request for tablet 075bc2aa8bad45298a553c8a315188a7: Uninitialized: clean time has not yet been initialized
I20260619 02:29:27.210212 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 18427
I20260619 02:29:27.217377 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 18561
I20260619 02:29:27.223810 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 18695
I20260619 02:29:27.230423 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 18333
2026-06-19T02:29:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2832 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:29:27 dist-test-slave-4qw6 krb5kdc[18866](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:29:27 dist-test-slave-4qw6 krb5kdc[18866](info): set up 2 sockets
Jun 19 02:29:27 dist-test-slave-4qw6 krb5kdc[18866](info): commencing operation
krb5kdc: starting...
W20260619 02:29:28.297581 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:29:28 dist-test-slave-4qw6 krb5kdc[18866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:29:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:29:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:29:28.427205 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:39071
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:34783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:39071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:28.546818 18879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:28.547144 18879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:28.547189 18879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:28.551672 18879 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:29:28.551767 18879 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 02:29:28.551785 18879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:28.551831 18879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:28.551869 18879 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:29:28.551898 18879 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:29:28.557710 18879 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:34783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:39071
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:39071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:28.559051 18879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:28.560195 18879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:28.569568 18885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:28.569568 18884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:28.569583 18887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:28.570139 18879 server_base.cc:1034] running on GCE node
I20260619 02:29:28.570828 18879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:28.572088 18879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:28.573321 18879 hybrid_clock.cc:648] HybridClock initialized: now 1781836168573309 us; error 35 us; skew 500 ppm
Jun 19 02:29:28 dist-test-slave-4qw6 krb5kdc[18866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836168, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:28.576355 18879 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:29:28.577316 18879 webserver.cc:460] Webserver started at http://127.17.96.254:44979/ using document root <none> and password file <none>
I20260619 02:29:28.577963 18879 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:28.578033 18879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:28.578222 18879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:28.580377 18879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "ab5a9f80d85747c0ae1a978733444848"
format_stamp: "Formatted at 2026-06-19 02:29:28 on dist-test-slave-4qw6"
server_key: "5ca0c009836acb401dd6a1ecd285214f"
server_key_iv: "0d42272050c65cf5b4d774ebb6e0796e"
server_key_version: "encryptionkey@0"
I20260619 02:29:28.580916 18879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "ab5a9f80d85747c0ae1a978733444848"
format_stamp: "Formatted at 2026-06-19 02:29:28 on dist-test-slave-4qw6"
server_key: "5ca0c009836acb401dd6a1ecd285214f"
server_key_iv: "0d42272050c65cf5b4d774ebb6e0796e"
server_key_version: "encryptionkey@0"
I20260619 02:29:28.584874 18879 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260619 02:29:28.587477 18894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:28.588790 18879 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 02:29:28.588905 18879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "ab5a9f80d85747c0ae1a978733444848"
format_stamp: "Formatted at 2026-06-19 02:29:28 on dist-test-slave-4qw6"
server_key: "5ca0c009836acb401dd6a1ecd285214f"
server_key_iv: "0d42272050c65cf5b4d774ebb6e0796e"
server_key_version: "encryptionkey@0"
I20260619 02:29:28.589007 18879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:28.602530 18879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:28.606061 18879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:28.606251 18879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:28.614884 18879 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:39071
I20260619 02:29:28.614954 18956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:39071 every 8 connection(s)
I20260619 02:29:28.615975 18879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:29:28.619055 18957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:28.624571 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 18879
I20260619 02:29:28.624686 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:29:28.624958 17795 external_mini_cluster.cc:1428] Setting key 768aea23a940e16a37fc8bc6f8af0b65
I20260619 02:29:28.625876 18957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848: Bootstrap starting.
I20260619 02:29:28.628960 18957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:28.629968 18957 log.cc:824] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:28.632417 18957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848: No bootstrap required, opened a new log
Jun 19 02:29:28 dist-test-slave-4qw6 krb5kdc[18866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:28.637228 18957 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5a9f80d85747c0ae1a978733444848" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 39071 } }
I20260619 02:29:28.637598 18957 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:28.637655 18957 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab5a9f80d85747c0ae1a978733444848, State: Initialized, Role: FOLLOWER
I20260619 02:29:28.638263 18957 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5a9f80d85747c0ae1a978733444848" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 39071 } }
I20260619 02:29:28.638411 18957 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:29:28.638537 18957 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:29:28.638680 18957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:28.639688 18957 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5a9f80d85747c0ae1a978733444848" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 39071 } }
I20260619 02:29:28.640138 18957 leader_election.cc:302] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab5a9f80d85747c0ae1a978733444848; no voters:
I20260619 02:29:28.640512 18957 leader_election.cc:288] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:29:28.640682 18962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:28.640929 18962 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848 [term 1 LEADER]: Becoming Leader. State: Replica: ab5a9f80d85747c0ae1a978733444848, State: Running, Role: LEADER
I20260619 02:29:28.641317 18962 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5a9f80d85747c0ae1a978733444848" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 39071 } }
I20260619 02:29:28.641471 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:28.626440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45928 (local address 127.17.96.254:39071)
0619 02:29:28.626911 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:28.626931 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:28.626974 (+ 43us) server_negotiation.cc:407] Connection header received
0619 02:29:28.627925 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:28.627952 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:28.628413 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:28.628822 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:28.629662 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:28.631064 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:28.631964 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:28.632748 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:28.632941 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:28.635688 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:28.635696 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:28.635716 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:28.635758 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:28.638026 (+ 2268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:28.638560 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:28.638566 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:28.638571 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:28.638756 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:28.639077 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:28.639081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:28.639083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:28.639538 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:28.639769 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:28.640073 (+ 304us) server_negotiation.cc:299] Negotiation successful
0619 02:29:28.640419 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":145,"threads_started":1}
I20260619 02:29:28.641937 18957 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:29:28.643118 18963 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab5a9f80d85747c0ae1a978733444848" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5a9f80d85747c0ae1a978733444848" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 39071 } } }
I20260619 02:29:28.643260 18963 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:28.643601 18964 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab5a9f80d85747c0ae1a978733444848. Latest consensus state: current_term: 1 leader_uuid: "ab5a9f80d85747c0ae1a978733444848" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5a9f80d85747c0ae1a978733444848" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 39071 } } }
I20260619 02:29:28.643719 18964 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:28.644320 18968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:29:28.647485 18968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:29:28.653296 18968 catalog_manager.cc:1347] Generated new cluster ID: 1f02ba4ac30149128721a0786d672cfa
I20260619 02:29:28.653391 18968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:29:28.677301 18968 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:29:28.678826 18968 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:29:28.693008 18968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848: Generated new TSK 0
I20260619 02:29:28.693972 18968 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.96.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
I20260619 02:29:28.729358 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:39071
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:34783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:28.851824 18983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:28.852137 18983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:28.852187 18983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:28.856734 18983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:28.856814 18983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:28.856858 18983 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:29:28.856894 18983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:29:28.862664 18983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:34783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.96.254:39071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:28.863826 18983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:28.864869 18983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:28.875490 18989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:28.875514 18991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:28.875514 18988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:28.875835 18983 server_base.cc:1034] running on GCE node
I20260619 02:29:28.876343 18983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:28.877081 18983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:28.878263 18983 hybrid_clock.cc:648] HybridClock initialized: now 1781836168878241 us; error 35 us; skew 500 ppm
Jun 19 02:29:28 dist-test-slave-4qw6 krb5kdc[18866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836168, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:28.881371 18983 init.cc:377] Logged in from keytab as kudu/127.17.96.193@KRBTEST.COM (short username kudu)
I20260619 02:29:28.882436 18983 webserver.cc:460] Webserver started at http://127.17.96.193:46631/ using document root <none> and password file <none>
I20260619 02:29:28.883020 18983 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:28.883086 18983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:28.883277 18983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:28.885442 18983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "5a774b506c294e4fbe76540f6ebf2f2a"
format_stamp: "Formatted at 2026-06-19 02:29:28 on dist-test-slave-4qw6"
server_key: "a48f255921f90776f4b1e04df154d2fc"
server_key_iv: "20e72d3a7dc7ef97ad9ab4344ad1b725"
server_key_version: "encryptionkey@0"
I20260619 02:29:28.886075 18983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "5a774b506c294e4fbe76540f6ebf2f2a"
format_stamp: "Formatted at 2026-06-19 02:29:28 on dist-test-slave-4qw6"
server_key: "a48f255921f90776f4b1e04df154d2fc"
server_key_iv: "20e72d3a7dc7ef97ad9ab4344ad1b725"
server_key_version: "encryptionkey@0"
I20260619 02:29:28.890149 18983 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 02:29:28.892874 18998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:28.894358 18983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:29:28.894506 18983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "5a774b506c294e4fbe76540f6ebf2f2a"
format_stamp: "Formatted at 2026-06-19 02:29:28 on dist-test-slave-4qw6"
server_key: "a48f255921f90776f4b1e04df154d2fc"
server_key_iv: "20e72d3a7dc7ef97ad9ab4344ad1b725"
server_key_version: "encryptionkey@0"
I20260619 02:29:28.894636 18983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:28.912233 18983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:28.915969 18983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:28.916206 18983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:28.917882 18983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:28.917955 18983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:28.918025 18983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:28.918078 18983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:29:28 dist-test-slave-4qw6 krb5kdc[18866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836168, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:28.929677 18983 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:37789
I20260619 02:29:28.929736 19113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:37789 every 8 connection(s)
I20260619 02:29:28.930940 18983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:29:28.933647 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:28.918677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:46569 (local address 127.17.96.254:39071)
0619 02:29:28.918860 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:28.918867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:28.920273 (+ 1406us) server_negotiation.cc:407] Connection header received
0619 02:29:28.921419 (+ 1146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:28.921422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:28.921485 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:28.921582 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:28.923499 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:28.924221 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:28.925996 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:28.926411 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:28.926505 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:28.929937 (+ 3432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:28.929942 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:28.929945 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:28.929978 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:28.931635 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:28.932210 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:28.932212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:28.932214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:28.932306 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:28.932626 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:28.932628 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:28.932630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:28.932837 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:28.932916 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:28.933394 (+ 478us) server_negotiation.cc:299] Negotiation successful
0619 02:29:28.933508 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 02:29:28.934540 19008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:28.919615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:39071 (local address 127.17.96.193:46569)
0619 02:29:28.920099 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:28.920162 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:28.921183 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:28.921817 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:28.921831 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:28.922434 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:28.923288 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:28.923314 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:28.924368 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:28.924372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:28.925824 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:28.925837 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:28.926620 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:28.926626 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:28.926889 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:28.927729 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:28.927755 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:28.929703 (+ 1948us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:28.931794 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:28.931800 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:28.931816 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:28.932093 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:28.932392 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:28.932394 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:28.932396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:28.932533 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:28.932925 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:28.932930 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:28.933178 (+ 248us) client_negotiation.cc:769] Sending connection context
0619 02:29:28.933335 (+ 157us) client_negotiation.cc:240] Negotiation successful
0619 02:29:28.933622 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":141,"threads_started":1}
I20260619 02:29:28.936071 19114 heartbeater.cc:347] Connected to a master server at 127.17.96.254:39071
I20260619 02:29:28.936282 19114 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:28.936869 19114 heartbeater.cc:511] Master 127.17.96.254:39071 requested a full tablet report, sending...
I20260619 02:29:28.937143 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 18983
I20260619 02:29:28.937255 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:29:28.937553 17795 external_mini_cluster.cc:1428] Setting key 8ea50f730bd32d5cde9bca67db7ef8d6
I20260619 02:29:28.938832 18910 ts_manager.cc:194] Registered new tserver with Master: 5a774b506c294e4fbe76540f6ebf2f2a (127.17.96.193:37789)
I20260619 02:29:28.940186 18910 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:46569
I20260619 02:29:28.947337 19120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:28.938436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:39071 (local address 127.17.96.193:43089)
0619 02:29:28.938778 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:28.938794 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:28.938907 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:28.939306 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:28.939309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:28.939550 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:28.939696 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:28.939708 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:28.940710 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:28.940716 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:28.941809 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:28.941825 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:28.942361 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:28.942366 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:28.942466 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:28.943032 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:28.943044 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:28.943470 (+ 426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:28.945665 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:28.945668 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:28.945669 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:28.945946 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:28.946257 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:28.946259 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:28.946261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:28.946337 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:28.946849 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:28.946851 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:28.946935 (+ 84us) client_negotiation.cc:769] Sending connection context
0619 02:29:28.947018 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 02:29:28.947166 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":89,"thread_start_us":111,"threads_started":1}
I20260619 02:29:28.947505 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:28.938524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:43089 (local address 127.17.96.254:39071)
0619 02:29:28.938634 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:28.938638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:28.938884 (+ 246us) server_negotiation.cc:407] Connection header received
0619 02:29:28.939028 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:28.939031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:28.939080 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:28.939149 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:28.939840 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:28.940552 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:28.941933 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:28.942236 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:28.942353 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:28.943655 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:28.943662 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:28.943668 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:28.943700 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:28.945539 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:28.946048 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:28.946052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:28.946054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:28.946157 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:28.946435 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:28.946441 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:28.946446 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:28.946747 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:28.946868 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:28.947115 (+ 247us) server_negotiation.cc:299] Negotiation successful
0619 02:29:28.947263 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
WARNING: no policy specified for kudu/127.17.96.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
I20260619 02:29:28.969580 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:39071
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:34783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:29.092135 19123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:29.092521 19123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:29.092638 19123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:29.097500 19123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:29.097582 19123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:29.097683 19123 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:29:29.097806 19123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:29:29.104203 19123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:34783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.96.254:39071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:29.105499 19123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:29.106699 19123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:29.117343 19128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:29.117429 19129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:29.117414 19123 server_base.cc:1034] running on GCE node
W20260619 02:29:29.117586 19131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:29.118104 19123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:29.118829 19123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:29.120023 19123 hybrid_clock.cc:648] HybridClock initialized: now 1781836169119992 us; error 53 us; skew 500 ppm
Jun 19 02:29:29 dist-test-slave-4qw6 krb5kdc[18866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836169, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:29.122964 19123 init.cc:377] Logged in from keytab as kudu/127.17.96.194@KRBTEST.COM (short username kudu)
I20260619 02:29:29.123898 19123 webserver.cc:460] Webserver started at http://127.17.96.194:34633/ using document root <none> and password file <none>
I20260619 02:29:29.124495 19123 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:29.124588 19123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:29.124831 19123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:29.127023 19123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "a8673b38caf44739ae8e3d85aa842afb"
format_stamp: "Formatted at 2026-06-19 02:29:29 on dist-test-slave-4qw6"
server_key: "9c9d2efe55cd45056e7e7a449c865c98"
server_key_iv: "2ba694126c8d680f33c284ba4b69e6ad"
server_key_version: "encryptionkey@0"
I20260619 02:29:29.127627 19123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "a8673b38caf44739ae8e3d85aa842afb"
format_stamp: "Formatted at 2026-06-19 02:29:29 on dist-test-slave-4qw6"
server_key: "9c9d2efe55cd45056e7e7a449c865c98"
server_key_iv: "2ba694126c8d680f33c284ba4b69e6ad"
server_key_version: "encryptionkey@0"
I20260619 02:29:29.131604 19123 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:29:29.134245 19138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:29.135573 19123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:29:29.135741 19123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "a8673b38caf44739ae8e3d85aa842afb"
format_stamp: "Formatted at 2026-06-19 02:29:29 on dist-test-slave-4qw6"
server_key: "9c9d2efe55cd45056e7e7a449c865c98"
server_key_iv: "2ba694126c8d680f33c284ba4b69e6ad"
server_key_version: "encryptionkey@0"
I20260619 02:29:29.135871 19123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:29.155370 19123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:29.158913 19123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:29.159150 19123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:29.160631 19123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:29.160713 19123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:29.160786 19123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:29.160828 19123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:29:29 dist-test-slave-4qw6 krb5kdc[18866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836169, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:29.172489 19123 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:37913
I20260619 02:29:29.172618 19253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:37913 every 8 connection(s)
I20260619 02:29:29.173770 19123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:29:29.175807 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.161608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:51471 (local address 127.17.96.254:39071)
0619 02:29:29.161737 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:29.161781 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:29.162948 (+ 1167us) server_negotiation.cc:407] Connection header received
0619 02:29:29.164173 (+ 1225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:29.164177 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:29.164251 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:29.164356 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:29.166127 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.167009 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.168443 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.168875 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.169007 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.171709 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:29.171714 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:29.171721 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:29.171758 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:29.173619 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:29.174272 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:29.174275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:29.174277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:29.174374 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:29.174791 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:29.174796 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:29.174798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:29.175013 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:29.175106 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:29.175579 (+ 473us) server_negotiation.cc:299] Negotiation successful
0619 02:29:29.175683 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:29:29.176626 19147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.162276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:39071 (local address 127.17.96.194:51471)
0619 02:29:29.162782 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:29.162827 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:29.163848 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:29.164619 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:29.164626 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:29.165145 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:29.165971 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.165993 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.167149 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.167153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.168299 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.168310 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.169064 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.169069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.169257 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.169986 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:29.170003 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:29.171510 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:29.173799 (+ 2289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:29.173805 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:29.173821 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:29.174137 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:29.174497 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:29.174500 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:29.174502 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:29.174665 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:29.175112 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:29.175117 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:29.175351 (+ 234us) client_negotiation.cc:769] Sending connection context
0619 02:29:29.175512 (+ 161us) client_negotiation.cc:240] Negotiation successful
0619 02:29:29.175772 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":135,"threads_started":1}
I20260619 02:29:29.177044 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 19123
I20260619 02:29:29.177168 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:29:29.177429 17795 external_mini_cluster.cc:1428] Setting key b6b704d47fe76f2f4454506eb6ac76b2
I20260619 02:29:29.178176 19254 heartbeater.cc:347] Connected to a master server at 127.17.96.254:39071
I20260619 02:29:29.178319 19254 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:29.178910 19254 heartbeater.cc:511] Master 127.17.96.254:39071 requested a full tablet report, sending...
I20260619 02:29:29.180044 18911 ts_manager.cc:194] Registered new tserver with Master: a8673b38caf44739ae8e3d85aa842afb (127.17.96.194:37913)
I20260619 02:29:29.180847 18911 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:51471
WARNING: no policy specified for kudu/127.17.96.195@KRBTEST.COM; defaulting to no policy
I20260619 02:29:29.189134 19261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.180444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:39071 (local address 127.17.96.194:42587)
0619 02:29:29.180803 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:29.180822 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:29.180928 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:29.181330 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:29.181335 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:29.181610 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:29.181810 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.181819 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.182976 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.182983 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.184033 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.184044 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.184521 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.184525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.184658 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.185208 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:29.185229 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:29.185774 (+ 545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:29.187546 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:29.187552 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:29.187556 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:29.187818 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:29.188121 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:29.188126 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:29.188128 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:29.188233 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:29.188656 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:29.188658 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:29.188748 (+ 90us) client_negotiation.cc:769] Sending connection context
0619 02:29:29.188840 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 02:29:29.188989 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":1,"thread_start_us":108,"threads_started":1}
I20260619 02:29:29.189105 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.180488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:42587 (local address 127.17.96.254:39071)
0619 02:29:29.180610 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:29.180614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:29.180911 (+ 297us) server_negotiation.cc:407] Connection header received
0619 02:29:29.181056 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:29.181062 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:29.181119 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:29.181174 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:29.181969 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.182800 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.184174 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.184389 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.184471 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.185930 (+ 1459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:29.185933 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:29.185936 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:29.185959 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:29.187417 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:29.187913 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:29.187915 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:29.187917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:29.187989 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:29.188330 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:29.188332 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:29.188333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:29.188563 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:29.188644 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:29.188875 (+ 231us) server_negotiation.cc:299] Negotiation successful
0619 02:29:29.188974 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
I20260619 02:29:29.208587 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:39071
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:34783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:29.324468 19263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:29.324807 19263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:29.324903 19263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:29.329231 19263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:29.329353 19263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:29.329437 19263 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:29:29.329521 19263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:29:29.335330 19263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:34783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.96.254:39071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:29.336621 19263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:29.337787 19263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:29.348431 19271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:29.348536 19263 server_base.cc:1034] running on GCE node
W20260619 02:29:29.348433 19269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:29.348449 19268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:29.349148 19263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:29.349886 19263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:29.351054 19263 hybrid_clock.cc:648] HybridClock initialized: now 1781836169351022 us; error 49 us; skew 500 ppm
Jun 19 02:29:29 dist-test-slave-4qw6 krb5kdc[18866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836169, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:29.353940 19263 init.cc:377] Logged in from keytab as kudu/127.17.96.195@KRBTEST.COM (short username kudu)
I20260619 02:29:29.354898 19263 webserver.cc:460] Webserver started at http://127.17.96.195:43267/ using document root <none> and password file <none>
I20260619 02:29:29.355545 19263 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:29.355621 19263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:29.355805 19263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:29.358004 19263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "56b6c80a9651431da62ceadd08e8151b"
format_stamp: "Formatted at 2026-06-19 02:29:29 on dist-test-slave-4qw6"
server_key: "de97e025bff788dfb3240dc418dceddd"
server_key_iv: "84ba886f7572a61a308f46bcd4630e6e"
server_key_version: "encryptionkey@0"
I20260619 02:29:29.358546 19263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "56b6c80a9651431da62ceadd08e8151b"
format_stamp: "Formatted at 2026-06-19 02:29:29 on dist-test-slave-4qw6"
server_key: "de97e025bff788dfb3240dc418dceddd"
server_key_iv: "84ba886f7572a61a308f46bcd4630e6e"
server_key_version: "encryptionkey@0"
I20260619 02:29:29.363068 19263 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 02:29:29.365691 19278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:29.367159 19263 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 02:29:29.367304 19263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "56b6c80a9651431da62ceadd08e8151b"
format_stamp: "Formatted at 2026-06-19 02:29:29 on dist-test-slave-4qw6"
server_key: "de97e025bff788dfb3240dc418dceddd"
server_key_iv: "84ba886f7572a61a308f46bcd4630e6e"
server_key_version: "encryptionkey@0"
I20260619 02:29:29.367431 19263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:29.396713 19263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:29.400543 19263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:29.400741 19263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:29.402417 19263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:29.402478 19263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:29.402522 19263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:29.402580 19263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:29:29 dist-test-slave-4qw6 krb5kdc[18866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836169, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:29.414002 19263 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:39809
I20260619 02:29:29.414064 19393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:39809 every 8 connection(s)
I20260619 02:29:29.415261 19263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:29:29.415810 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 19263
I20260619 02:29:29.415930 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:29:29.416245 17795 external_mini_cluster.cc:1428] Setting key f4bdca0f95dda2f5990e27ee32f6c7f7
I20260619 02:29:29.417330 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.403262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:36851 (local address 127.17.96.254:39071)
0619 02:29:29.403389 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:29.403393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:29.404497 (+ 1104us) server_negotiation.cc:407] Connection header received
0619 02:29:29.405687 (+ 1190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:29.405693 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:29.405788 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:29.405896 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:29.407321 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.408239 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.409614 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.409942 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.410067 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.412973 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:29.412976 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:29.412978 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:29.413005 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:29.414846 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:29.415435 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:29.415439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:29.415441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:29.415565 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:29.416023 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:29.416030 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:29.416032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:29.416286 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:29.416369 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:29.417025 (+ 656us) server_negotiation.cc:299] Negotiation successful
0619 02:29:29.417158 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 02:29:29.418234 19287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.403854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:39071 (local address 127.17.96.195:36851)
0619 02:29:29.404342 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:29.404386 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:29.405384 (+ 998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:29.406122 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:29.406133 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:29.406514 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:29.407115 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.407132 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.408389 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.408393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.409484 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.409494 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.410122 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.410126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.410306 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.410938 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:29.410959 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:29.412779 (+ 1820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:29.414999 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:29.415004 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:29.415022 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:29.415287 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:29.415700 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:29.415706 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:29.415712 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:29.415892 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:29.416446 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:29.416453 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:29.416748 (+ 295us) client_negotiation.cc:769] Sending connection context
0619 02:29:29.416955 (+ 207us) client_negotiation.cc:240] Negotiation successful
0619 02:29:29.417317 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":131,"threads_started":1}
I20260619 02:29:29.419699 19394 heartbeater.cc:347] Connected to a master server at 127.17.96.254:39071
I20260619 02:29:29.419878 19394 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:29.420423 19394 heartbeater.cc:511] Master 127.17.96.254:39071 requested a full tablet report, sending...
I20260619 02:29:29.421530 18910 ts_manager.cc:194] Registered new tserver with Master: 56b6c80a9651431da62ceadd08e8151b (127.17.96.195:39809)
I20260619 02:29:29.422344 18910 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:36851
I20260619 02:29:29.428611 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.421925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:39923 (local address 127.17.96.254:39071)
0619 02:29:29.422041 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:29.422046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:29.422224 (+ 178us) server_negotiation.cc:407] Connection header received
0619 02:29:29.422308 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:29.422311 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:29.422361 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:29.422454 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:29.423032 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.423762 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.424579 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.424790 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.424871 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.426045 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:29.426048 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:29.426050 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:29.426070 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:29.427278 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:29.427660 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:29.427662 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:29.427663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:29.427732 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:29.427960 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:29.427962 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:29.427963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:29.428123 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:29.428196 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:29.428407 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 02:29:29.428499 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 02:29:29.428608 19400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.421864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:39071 (local address 127.17.96.195:39923)
0619 02:29:29.422143 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:29.422156 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:29.422240 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:29.422530 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:29.422533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:29.422775 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:29.422909 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.422918 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.423894 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.423896 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.424462 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.424470 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.424892 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.424894 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.424966 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.425527 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:29.425539 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:29.425932 (+ 393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:29.427375 (+ 1443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:29.427377 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:29.427379 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:29.427571 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:29.427807 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:29.427809 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:29.427810 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:29.427888 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:29.428199 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:29.428201 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:29.428275 (+ 74us) client_negotiation.cc:769] Sending connection context
0619 02:29:29.428352 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 02:29:29.428462 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":112,"threads_started":1}
I20260619 02:29:29.430421 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 02:29:29 dist-test-slave-4qw6 krb5kdc[18866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836169, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 02:29:29 dist-test-slave-4qw6 krb5kdc[18866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836169, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:29.456774 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.448695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45940 (local address 127.17.96.254:39071)
0619 02:29:29.448827 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:29.448830 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:29.448845 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:29:29.448889 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:29.448908 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:29.448949 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:29.449021 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:29.449699 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.450299 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.451103 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.451423 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.451506 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.454057 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:29.454060 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:29.454062 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:29.454086 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:29.455366 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:29.455748 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:29.455751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:29.455752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:29.455828 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:29.456064 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:29.456066 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:29.456067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:29.456239 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:29.456338 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:29.456553 (+ 215us) server_negotiation.cc:299] Negotiation successful
0619 02:29:29.456649 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 02:29:29.460089 18910 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:29:29.463354 18910 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:29:29.479125 19414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.472527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:39809 (local address 127.0.0.1:46180)
0619 02:29:29.473431 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:29.473487 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:29.473626 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:29.474426 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:29.474432 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:29.474465 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:29.474784 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.474792 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.476025 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.476031 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.477316 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.477327 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.478525 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.478529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.478638 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.478693 (+ 55us) client_negotiation.cc:769] Sending connection context
0619 02:29:29.478811 (+ 118us) client_negotiation.cc:240] Negotiation successful
0619 02:29:29.478938 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"mutex_wait_us":107,"spinlock_wait_cycles":7040,"thread_start_us":89,"threads_started":1}
I20260619 02:29:29.479166 19412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.472661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46180 (local address 127.17.96.195:39809)
0619 02:29:29.473903 (+ 1242us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:29.473915 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:29.473941 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:29:29.474029 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:29.474035 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:29.474252 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:29.474428 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:29.474910 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.475895 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.477455 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.478396 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.478528 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.478730 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:29.478865 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 02:29:29.478986 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1140,"thread_start_us":139,"threads_started":1}
I20260619 02:29:29.479193 19411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.472675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:37789 (local address 127.0.0.1:39154)
0619 02:29:29.473888 (+ 1213us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:29.473898 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:29.473955 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:29.474426 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:29.474432 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:29.474467 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:29.474785 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.474792 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.476177 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.476180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.477446 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.477456 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.478715 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.478720 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.478865 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.478895 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 02:29:29.478966 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 02:29:29.479034 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1153,"mutex_wait_us":104,"spinlock_wait_cycles":5632,"thread_start_us":133,"threads_started":1}
I20260619 02:29:29.479821 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.473374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35818 (local address 127.17.96.194:37913)
0619 02:29:29.473822 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:29.473838 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:29.474809 (+ 971us) server_negotiation.cc:407] Connection header received
0619 02:29:29.475026 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:29.475032 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:29.475232 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:29.475383 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:29.475663 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.476747 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.478091 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.478932 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.479024 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.479180 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:29.479592 (+ 412us) server_negotiation.cc:299] Negotiation successful
0619 02:29:29.479686 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":96,"threads_started":1}
I20260619 02:29:29.480324 19415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.473366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39154 (local address 127.17.96.193:37789)
0619 02:29:29.474004 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:29.474011 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:29.474031 (+ 20us) server_negotiation.cc:407] Connection header received
0619 02:29:29.474096 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:29.474100 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:29.474252 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:29.474428 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:29.474950 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.476044 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.477606 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.478419 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.478528 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.479795 (+ 1267us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:29.479853 (+ 58us) server_negotiation.cc:299] Negotiation successful
0619 02:29:29.480153 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":98,"threads_started":1}
I20260619 02:29:29.481128 19413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.472966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:37913 (local address 127.0.0.1:35818)
0619 02:29:29.474669 (+ 1703us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:29.474682 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:29.474818 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:29.475395 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:29.475399 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:29.475419 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:29.475534 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.475541 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.476875 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.476878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.477966 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.477974 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.479244 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.479249 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.479417 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.479444 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 02:29:29.480012 (+ 568us) client_negotiation.cc:240] Negotiation successful
0619 02:29:29.480092 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1649,"thread_start_us":88,"threads_started":1}
I20260619 02:29:29.482465 19328 tablet_service.cc:1467] Processing CreateTablet for tablet 630602b043e44feaa0800949e2edb749 (DEFAULT_TABLE table=test-table [id=d843bdf230f04d08ab4a66069b3857ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:29.482937 19048 tablet_service.cc:1467] Processing CreateTablet for tablet 630602b043e44feaa0800949e2edb749 (DEFAULT_TABLE table=test-table [id=d843bdf230f04d08ab4a66069b3857ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:29.483708 19328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 630602b043e44feaa0800949e2edb749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:29.483889 19048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 630602b043e44feaa0800949e2edb749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:29.483546 19188 tablet_service.cc:1467] Processing CreateTablet for tablet 630602b043e44feaa0800949e2edb749 (DEFAULT_TABLE table=test-table [id=d843bdf230f04d08ab4a66069b3857ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:29.484549 19188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 630602b043e44feaa0800949e2edb749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:29.494058 19417 tablet_bootstrap.cc:492] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b: Bootstrap starting.
I20260619 02:29:29.495599 19418 tablet_bootstrap.cc:492] T 630602b043e44feaa0800949e2edb749 P 5a774b506c294e4fbe76540f6ebf2f2a: Bootstrap starting.
I20260619 02:29:29.496686 19417 tablet_bootstrap.cc:654] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:29.497506 19418 tablet_bootstrap.cc:654] T 630602b043e44feaa0800949e2edb749 P 5a774b506c294e4fbe76540f6ebf2f2a: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:29.497768 19417 log.cc:824] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:29.498348 19418 log.cc:824] T 630602b043e44feaa0800949e2edb749 P 5a774b506c294e4fbe76540f6ebf2f2a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:29.500253 19418 tablet_bootstrap.cc:492] T 630602b043e44feaa0800949e2edb749 P 5a774b506c294e4fbe76540f6ebf2f2a: No bootstrap required, opened a new log
I20260619 02:29:29.500545 19418 ts_tablet_manager.cc:1397] T 630602b043e44feaa0800949e2edb749 P 5a774b506c294e4fbe76540f6ebf2f2a: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260619 02:29:29.503412 19417 tablet_bootstrap.cc:492] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b: No bootstrap required, opened a new log
I20260619 02:29:29.504698 19417 ts_tablet_manager.cc:1397] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260619 02:29:29.505043 19418 raft_consensus.cc:348] T 630602b043e44feaa0800949e2edb749 P 5a774b506c294e4fbe76540f6ebf2f2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } } peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } }
I20260619 02:29:29.505388 19418 raft_consensus.cc:374] T 630602b043e44feaa0800949e2edb749 P 5a774b506c294e4fbe76540f6ebf2f2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:29.505443 19418 raft_consensus.cc:729] T 630602b043e44feaa0800949e2edb749 P 5a774b506c294e4fbe76540f6ebf2f2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a774b506c294e4fbe76540f6ebf2f2a, State: Initialized, Role: FOLLOWER
I20260619 02:29:29.505589 19421 tablet_bootstrap.cc:492] T 630602b043e44feaa0800949e2edb749 P a8673b38caf44739ae8e3d85aa842afb: Bootstrap starting.
I20260619 02:29:29.506009 19418 consensus_queue.cc:260] T 630602b043e44feaa0800949e2edb749 P 5a774b506c294e4fbe76540f6ebf2f2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } } peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } }
I20260619 02:29:29.507068 19418 ts_tablet_manager.cc:1428] T 630602b043e44feaa0800949e2edb749 P 5a774b506c294e4fbe76540f6ebf2f2a: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260619 02:29:29.507484 19114 heartbeater.cc:503] Master 127.17.96.254:39071 was elected leader, sending a full tablet report...
I20260619 02:29:29.508756 19421 tablet_bootstrap.cc:654] T 630602b043e44feaa0800949e2edb749 P a8673b38caf44739ae8e3d85aa842afb: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:29.509599 19417 raft_consensus.cc:348] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } } peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } }
I20260619 02:29:29.509900 19421 log.cc:824] T 630602b043e44feaa0800949e2edb749 P a8673b38caf44739ae8e3d85aa842afb: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:29.510064 19417 raft_consensus.cc:374] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:29.510166 19417 raft_consensus.cc:729] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56b6c80a9651431da62ceadd08e8151b, State: Initialized, Role: FOLLOWER
I20260619 02:29:29.511124 19417 consensus_queue.cc:260] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } } peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } }
I20260619 02:29:29.512286 19417 ts_tablet_manager.cc:1428] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260619 02:29:29.512523 19421 tablet_bootstrap.cc:492] T 630602b043e44feaa0800949e2edb749 P a8673b38caf44739ae8e3d85aa842afb: No bootstrap required, opened a new log
I20260619 02:29:29.512652 19394 heartbeater.cc:503] Master 127.17.96.254:39071 was elected leader, sending a full tablet report...
I20260619 02:29:29.512787 19421 ts_tablet_manager.cc:1397] T 630602b043e44feaa0800949e2edb749 P a8673b38caf44739ae8e3d85aa842afb: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260619 02:29:29.518326 19421 raft_consensus.cc:348] T 630602b043e44feaa0800949e2edb749 P a8673b38caf44739ae8e3d85aa842afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } } peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } }
I20260619 02:29:29.518769 19421 raft_consensus.cc:374] T 630602b043e44feaa0800949e2edb749 P a8673b38caf44739ae8e3d85aa842afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:29.518852 19421 raft_consensus.cc:729] T 630602b043e44feaa0800949e2edb749 P a8673b38caf44739ae8e3d85aa842afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8673b38caf44739ae8e3d85aa842afb, State: Initialized, Role: FOLLOWER
I20260619 02:29:29.519464 19421 consensus_queue.cc:260] T 630602b043e44feaa0800949e2edb749 P a8673b38caf44739ae8e3d85aa842afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } } peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } }
I20260619 02:29:29.520535 19254 heartbeater.cc:503] Master 127.17.96.254:39071 was elected leader, sending a full tablet report...
I20260619 02:29:29.520882 19421 ts_tablet_manager.cc:1428] T 630602b043e44feaa0800949e2edb749 P a8673b38caf44739ae8e3d85aa842afb: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
I20260619 02:29:29.530490 19424 raft_consensus.cc:482] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:29:29.530723 19424 raft_consensus.cc:504] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } } peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } }
I20260619 02:29:29.532135 19424 leader_election.cc:288] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a774b506c294e4fbe76540f6ebf2f2a (127.17.96.193:37789), a8673b38caf44739ae8e3d85aa842afb (127.17.96.194:37913)
I20260619 02:29:29.537662 19415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.532569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:52309 (local address 127.17.96.193:37789)
0619 02:29:29.532704 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:29.532709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:29.532725 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:29:29.532936 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:29.532942 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:29.533011 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:29.533091 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:29.533634 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.534238 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.536563 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.537064 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.537188 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.537252 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:29.537449 (+ 197us) server_negotiation.cc:299] Negotiation successful
0619 02:29:29.537534 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 02:29:29.537662 19287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.532388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:37789 (local address 127.17.96.195:52309)
0619 02:29:29.532646 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:29.532666 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:29.532769 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:29.533240 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:29.533245 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:29.533266 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:29.533486 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.533496 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.534357 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.534362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.536400 (+ 2038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.536412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.537188 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.537192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.537325 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.537344 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 02:29:29.537433 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 02:29:29.537517 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":186}
I20260619 02:29:29.538342 19426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.532461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:37913 (local address 127.17.96.195:41753)
0619 02:29:29.532928 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:29.532943 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:29.533040 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:29.533296 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:29.533298 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:29.533314 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:29.533512 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.533518 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.534794 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.534798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.536400 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.536412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.537890 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.537896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.538020 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.538046 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 02:29:29.538120 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 02:29:29.538192 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":166,"thread_start_us":150,"threads_started":1}
I20260619 02:29:29.538475 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.532929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:41753 (local address 127.17.96.194:37913)
0619 02:29:29.533072 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:29.533078 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:29.533107 (+ 29us) server_negotiation.cc:407] Connection header received
0619 02:29:29.533164 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:29.533166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:29.533203 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:29.533285 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:29.533708 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.534658 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.536542 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.537422 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.537577 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.537650 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:29.538170 (+ 520us) server_negotiation.cc:299] Negotiation successful
0619 02:29:29.538284 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 02:29:29.539017 19068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "630602b043e44feaa0800949e2edb749" candidate_uuid: "56b6c80a9651431da62ceadd08e8151b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" is_pre_election: true
I20260619 02:29:29.539075 19208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "630602b043e44feaa0800949e2edb749" candidate_uuid: "56b6c80a9651431da62ceadd08e8151b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8673b38caf44739ae8e3d85aa842afb" is_pre_election: true
I20260619 02:29:29.539376 19068 raft_consensus.cc:2454] T 630602b043e44feaa0800949e2edb749 P 5a774b506c294e4fbe76540f6ebf2f2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b6c80a9651431da62ceadd08e8151b in term 0.
I20260619 02:29:29.539404 19208 raft_consensus.cc:2454] T 630602b043e44feaa0800949e2edb749 P a8673b38caf44739ae8e3d85aa842afb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b6c80a9651431da62ceadd08e8151b in term 0.
I20260619 02:29:29.539955 19281 leader_election.cc:302] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 56b6c80a9651431da62ceadd08e8151b, 5a774b506c294e4fbe76540f6ebf2f2a, a8673b38caf44739ae8e3d85aa842afb; no voters:
I20260619 02:29:29.540266 19424 raft_consensus.cc:2781] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:29:29.540388 19424 raft_consensus.cc:482] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:29:29.540448 19424 raft_consensus.cc:3037] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:29.542286 19424 raft_consensus.cc:504] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } } peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } }
I20260619 02:29:29.542836 19424 leader_election.cc:288] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b [CANDIDATE]: Term 1 election: Requested vote from peers 5a774b506c294e4fbe76540f6ebf2f2a (127.17.96.193:37789), a8673b38caf44739ae8e3d85aa842afb (127.17.96.194:37913)
I20260619 02:29:29.543215 19068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "630602b043e44feaa0800949e2edb749" candidate_uuid: "56b6c80a9651431da62ceadd08e8151b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a774b506c294e4fbe76540f6ebf2f2a"
I20260619 02:29:29.543267 19208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "630602b043e44feaa0800949e2edb749" candidate_uuid: "56b6c80a9651431da62ceadd08e8151b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8673b38caf44739ae8e3d85aa842afb"
I20260619 02:29:29.543397 19068 raft_consensus.cc:3037] T 630602b043e44feaa0800949e2edb749 P 5a774b506c294e4fbe76540f6ebf2f2a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:29.543435 19208 raft_consensus.cc:3037] T 630602b043e44feaa0800949e2edb749 P a8673b38caf44739ae8e3d85aa842afb [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:29.544792 19068 raft_consensus.cc:2454] T 630602b043e44feaa0800949e2edb749 P 5a774b506c294e4fbe76540f6ebf2f2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b6c80a9651431da62ceadd08e8151b in term 1.
I20260619 02:29:29.545217 19280 leader_election.cc:302] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56b6c80a9651431da62ceadd08e8151b, 5a774b506c294e4fbe76540f6ebf2f2a; no voters:
I20260619 02:29:29.545490 19424 raft_consensus.cc:2781] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:29.545640 19208 raft_consensus.cc:2454] T 630602b043e44feaa0800949e2edb749 P a8673b38caf44739ae8e3d85aa842afb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b6c80a9651431da62ceadd08e8151b in term 1.
I20260619 02:29:29.545710 19424 raft_consensus.cc:686] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b [term 1 LEADER]: Becoming Leader. State: Replica: 56b6c80a9651431da62ceadd08e8151b, State: Running, Role: LEADER
I20260619 02:29:29.546203 19424 consensus_queue.cc:237] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } } peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } }
I20260619 02:29:29.549629 18910 catalog_manager.cc:5495] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b reported cstate change: term changed from 0 to 1, leader changed from <none> to 56b6c80a9651431da62ceadd08e8151b (127.17.96.195). New cstate: current_term: 1 leader_uuid: "56b6c80a9651431da62ceadd08e8151b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } health_report { overall_health: UNKNOWN } } }
I20260619 02:29:29.577394 19412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.572954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46188 (local address 127.17.96.195:39809)
0619 02:29:29.573086 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:29.573091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:29.573184 (+ 93us) server_negotiation.cc:407] Connection header received
0619 02:29:29.573435 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:29.573439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:29.573512 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:29.573621 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:29:29.573968 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.574501 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.575467 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.575886 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.576051 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.576443 (+ 392us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:29:29.576940 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:29:29.577079 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:29.577160 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 02:29:29.577228 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:29:29.584908 19208 raft_consensus.cc:1261] T 630602b043e44feaa0800949e2edb749 P a8673b38caf44739ae8e3d85aa842afb [term 1 FOLLOWER]: Refusing update from remote peer 56b6c80a9651431da62ceadd08e8151b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:29:29.584908 19068 raft_consensus.cc:1261] T 630602b043e44feaa0800949e2edb749 P 5a774b506c294e4fbe76540f6ebf2f2a [term 1 FOLLOWER]: Refusing update from remote peer 56b6c80a9651431da62ceadd08e8151b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:29:29.585685 19424 consensus_queue.cc:1035] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:29:29.585906 19427 consensus_queue.cc:1035] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:29:29.610078 18911 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:29:29.610307 18911 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:29:29.613685 18911 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848: Sending DeleteTablet for 3 replicas of tablet 630602b043e44feaa0800949e2edb749
I20260619 02:29:29.614930 19328 tablet_service.cc:1514] Processing DeleteTablet for tablet 630602b043e44feaa0800949e2edb749 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:29 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:46180
I20260619 02:29:29.614923 19188 tablet_service.cc:1514] Processing DeleteTablet for tablet 630602b043e44feaa0800949e2edb749 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:29 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:35818
I20260619 02:29:29.615785 19441 tablet_replica.cc:330] T 630602b043e44feaa0800949e2edb749 P a8673b38caf44739ae8e3d85aa842afb: stopping tablet replica
I20260619 02:29:29.615969 19048 tablet_service.cc:1514] Processing DeleteTablet for tablet 630602b043e44feaa0800949e2edb749 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:29 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:39154
I20260619 02:29:29.616066 19440 tablet_replica.cc:330] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b: stopping tablet replica
I20260619 02:29:29.617532 19441 raft_consensus.cc:2229] T 630602b043e44feaa0800949e2edb749 P a8673b38caf44739ae8e3d85aa842afb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:29:29.617717 18911 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 02:29:29.617996 19441 raft_consensus.cc:2258] T 630602b043e44feaa0800949e2edb749 P a8673b38caf44739ae8e3d85aa842afb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 02:29:29.618397 18911 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:29:29.617885 19440 raft_consensus.cc:2229] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:29:29.619038 19440 raft_consensus.cc:2258] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:29.620070 19441 ts_tablet_manager.cc:1905] T 630602b043e44feaa0800949e2edb749 P a8673b38caf44739ae8e3d85aa842afb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:29.621851 19440 ts_tablet_manager.cc:1905] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:29.624605 19328 tablet_service.cc:1467] Processing CreateTablet for tablet 8370b665876f48289ba655ce58a988ee (DEFAULT_TABLE table=test-table [id=be1ab3de38334045b8273a4cdd78621d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:29.624683 19188 tablet_service.cc:1467] Processing CreateTablet for tablet 8370b665876f48289ba655ce58a988ee (DEFAULT_TABLE table=test-table [id=be1ab3de38334045b8273a4cdd78621d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:29.624919 19188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8370b665876f48289ba655ce58a988ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:29.624919 19328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8370b665876f48289ba655ce58a988ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:29.625175 19440 ts_tablet_manager.cc:1918] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:29:29.625330 19440 log.cc:1196] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/630602b043e44feaa0800949e2edb749
I20260619 02:29:29.625774 19440 ts_tablet_manager.cc:1939] T 630602b043e44feaa0800949e2edb749 P 56b6c80a9651431da62ceadd08e8151b: Deleting consensus metadata
I20260619 02:29:29.627233 19417 tablet_bootstrap.cc:492] T 8370b665876f48289ba655ce58a988ee P 56b6c80a9651431da62ceadd08e8151b: Bootstrap starting.
I20260619 02:29:29.627328 19421 tablet_bootstrap.cc:492] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb: Bootstrap starting.
I20260619 02:29:29.628234 19417 tablet_bootstrap.cc:654] T 8370b665876f48289ba655ce58a988ee P 56b6c80a9651431da62ceadd08e8151b: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:29.628598 19421 tablet_bootstrap.cc:654] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:29.629026 18898 catalog_manager.cc:4841] TS 56b6c80a9651431da62ceadd08e8151b (127.17.96.195:39809): tablet 630602b043e44feaa0800949e2edb749 (table test-table [id=d843bdf230f04d08ab4a66069b3857ce]) successfully deleted
I20260619 02:29:29.629999 19442 tablet_replica.cc:330] T 630602b043e44feaa0800949e2edb749 P 5a774b506c294e4fbe76540f6ebf2f2a: stopping tablet replica
I20260619 02:29:29.630246 19048 tablet_service.cc:1467] Processing CreateTablet for tablet 8370b665876f48289ba655ce58a988ee (DEFAULT_TABLE table=test-table [id=be1ab3de38334045b8273a4cdd78621d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:29.630426 19442 raft_consensus.cc:2229] T 630602b043e44feaa0800949e2edb749 P 5a774b506c294e4fbe76540f6ebf2f2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:29:29.630630 19048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8370b665876f48289ba655ce58a988ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:29.630831 19442 raft_consensus.cc:2258] T 630602b043e44feaa0800949e2edb749 P 5a774b506c294e4fbe76540f6ebf2f2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:29.633569 19442 ts_tablet_manager.cc:1905] T 630602b043e44feaa0800949e2edb749 P 5a774b506c294e4fbe76540f6ebf2f2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:29.633940 19418 tablet_bootstrap.cc:492] T 8370b665876f48289ba655ce58a988ee P 5a774b506c294e4fbe76540f6ebf2f2a: Bootstrap starting.
I20260619 02:29:29.635273 19418 tablet_bootstrap.cc:654] T 8370b665876f48289ba655ce58a988ee P 5a774b506c294e4fbe76540f6ebf2f2a: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:29.637718 19442 ts_tablet_manager.cc:1918] T 630602b043e44feaa0800949e2edb749 P 5a774b506c294e4fbe76540f6ebf2f2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:29:29.637785 19441 ts_tablet_manager.cc:1918] T 630602b043e44feaa0800949e2edb749 P a8673b38caf44739ae8e3d85aa842afb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:29:29.637912 19441 log.cc:1196] T 630602b043e44feaa0800949e2edb749 P a8673b38caf44739ae8e3d85aa842afb: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/630602b043e44feaa0800949e2edb749
I20260619 02:29:29.637993 19442 log.cc:1196] T 630602b043e44feaa0800949e2edb749 P 5a774b506c294e4fbe76540f6ebf2f2a: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-0/wal/wals/630602b043e44feaa0800949e2edb749
I20260619 02:29:29.638294 19441 ts_tablet_manager.cc:1939] T 630602b043e44feaa0800949e2edb749 P a8673b38caf44739ae8e3d85aa842afb: Deleting consensus metadata
I20260619 02:29:29.638731 19442 ts_tablet_manager.cc:1939] T 630602b043e44feaa0800949e2edb749 P 5a774b506c294e4fbe76540f6ebf2f2a: Deleting consensus metadata
I20260619 02:29:29.640292 18896 catalog_manager.cc:4841] TS 5a774b506c294e4fbe76540f6ebf2f2a (127.17.96.193:37789): tablet 630602b043e44feaa0800949e2edb749 (table test-table [id=d843bdf230f04d08ab4a66069b3857ce]) successfully deleted
I20260619 02:29:29.640533 18897 catalog_manager.cc:4841] TS a8673b38caf44739ae8e3d85aa842afb (127.17.96.194:37913): tablet 630602b043e44feaa0800949e2edb749 (table test-table [id=d843bdf230f04d08ab4a66069b3857ce]) successfully deleted
I20260619 02:29:29.641573 19418 tablet_bootstrap.cc:492] T 8370b665876f48289ba655ce58a988ee P 5a774b506c294e4fbe76540f6ebf2f2a: No bootstrap required, opened a new log
I20260619 02:29:29.641661 19418 ts_tablet_manager.cc:1397] T 8370b665876f48289ba655ce58a988ee P 5a774b506c294e4fbe76540f6ebf2f2a: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.002s
I20260619 02:29:29.641942 19421 tablet_bootstrap.cc:492] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb: No bootstrap required, opened a new log
I20260619 02:29:29.642041 19421 ts_tablet_manager.cc:1397] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20260619 02:29:29.642462 19418 raft_consensus.cc:348] T 8370b665876f48289ba655ce58a988ee P 5a774b506c294e4fbe76540f6ebf2f2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } }
I20260619 02:29:29.642585 19418 raft_consensus.cc:374] T 8370b665876f48289ba655ce58a988ee P 5a774b506c294e4fbe76540f6ebf2f2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:29.642618 19418 raft_consensus.cc:729] T 8370b665876f48289ba655ce58a988ee P 5a774b506c294e4fbe76540f6ebf2f2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a774b506c294e4fbe76540f6ebf2f2a, State: Initialized, Role: FOLLOWER
I20260619 02:29:29.642747 19418 consensus_queue.cc:260] T 8370b665876f48289ba655ce58a988ee P 5a774b506c294e4fbe76540f6ebf2f2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } }
I20260619 02:29:29.642707 19421 raft_consensus.cc:348] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } }
I20260619 02:29:29.642843 19421 raft_consensus.cc:374] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:29.642884 19421 raft_consensus.cc:729] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8673b38caf44739ae8e3d85aa842afb, State: Initialized, Role: FOLLOWER
I20260619 02:29:29.643009 19418 ts_tablet_manager.cc:1428] T 8370b665876f48289ba655ce58a988ee P 5a774b506c294e4fbe76540f6ebf2f2a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260619 02:29:29.643028 19421 consensus_queue.cc:260] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } }
I20260619 02:29:29.643373 19421 ts_tablet_manager.cc:1428] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:29:29.647393 19417 tablet_bootstrap.cc:492] T 8370b665876f48289ba655ce58a988ee P 56b6c80a9651431da62ceadd08e8151b: No bootstrap required, opened a new log
I20260619 02:29:29.647527 19417 ts_tablet_manager.cc:1397] T 8370b665876f48289ba655ce58a988ee P 56b6c80a9651431da62ceadd08e8151b: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20260619 02:29:29.648049 19417 raft_consensus.cc:348] T 8370b665876f48289ba655ce58a988ee P 56b6c80a9651431da62ceadd08e8151b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } }
I20260619 02:29:29.648181 19417 raft_consensus.cc:374] T 8370b665876f48289ba655ce58a988ee P 56b6c80a9651431da62ceadd08e8151b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:29.648212 19417 raft_consensus.cc:729] T 8370b665876f48289ba655ce58a988ee P 56b6c80a9651431da62ceadd08e8151b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56b6c80a9651431da62ceadd08e8151b, State: Initialized, Role: FOLLOWER
I20260619 02:29:29.648356 19417 consensus_queue.cc:260] T 8370b665876f48289ba655ce58a988ee P 56b6c80a9651431da62ceadd08e8151b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } }
I20260619 02:29:29.648643 19417 ts_tablet_manager.cc:1428] T 8370b665876f48289ba655ce58a988ee P 56b6c80a9651431da62ceadd08e8151b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260619 02:29:29.666567 19395 tablet.cc:2369] T 8370b665876f48289ba655ce58a988ee P 56b6c80a9651431da62ceadd08e8151b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:29:29.675002 19255 tablet.cc:2369] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:29:29.682438 19115 tablet.cc:2369] T 8370b665876f48289ba655ce58a988ee P 5a774b506c294e4fbe76540f6ebf2f2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:29:29.698079 19425 raft_consensus.cc:482] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:29:29.698273 19425 raft_consensus.cc:504] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } }
I20260619 02:29:29.699878 19425 leader_election.cc:288] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a774b506c294e4fbe76540f6ebf2f2a (127.17.96.193:37789), 56b6c80a9651431da62ceadd08e8151b (127.17.96.195:39809)
I20260619 02:29:29.705526 19415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.700490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:54869 (local address 127.17.96.193:37789)
0619 02:29:29.700616 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:29.700621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:29.700947 (+ 326us) server_negotiation.cc:407] Connection header received
0619 02:29:29.701154 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:29.701158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:29.701205 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:29.701253 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:29.701647 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.702473 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.704127 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.704743 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.704857 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.704916 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:29.705264 (+ 348us) server_negotiation.cc:299] Negotiation successful
0619 02:29:29.705358 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:29:29.706220 19447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.700093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:37789 (local address 127.17.96.194:54869)
0619 02:29:29.700861 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:29.700878 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:29.701030 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:29.701397 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:29.701401 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:29.701418 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:29.701530 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.701536 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.702626 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.702629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.703978 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.703989 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.705014 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.705017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.705126 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.705150 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 02:29:29.705986 (+ 836us) client_negotiation.cc:240] Negotiation successful
0619 02:29:29.706085 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"thread_start_us":90,"threads_started":1}
I20260619 02:29:29.706624 19412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.700694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:59353 (local address 127.17.96.195:39809)
0619 02:29:29.700847 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:29.700853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:29.701164 (+ 311us) server_negotiation.cc:407] Connection header received
0619 02:29:29.701423 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:29.701428 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:29.701497 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:29.701557 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:29.702088 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.703024 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.704830 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.705634 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.705753 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.705875 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:29.706333 (+ 458us) server_negotiation.cc:299] Negotiation successful
0619 02:29:29.706446 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 02:29:29.706948 19068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8370b665876f48289ba655ce58a988ee" candidate_uuid: "a8673b38caf44739ae8e3d85aa842afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" is_pre_election: true
I20260619 02:29:29.707129 19068 raft_consensus.cc:2454] T 8370b665876f48289ba655ce58a988ee P 5a774b506c294e4fbe76540f6ebf2f2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8673b38caf44739ae8e3d85aa842afb in term 0.
I20260619 02:29:29.707170 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.700274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:39809 (local address 127.17.96.194:59353)
0619 02:29:29.701065 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:29.701082 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:29.701254 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:29.701729 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:29.701734 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:29.701793 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:29.701928 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.701937 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.703201 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.703205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.704657 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.704672 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.706001 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.706006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.706157 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.706190 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 02:29:29.706848 (+ 658us) client_negotiation.cc:240] Negotiation successful
0619 02:29:29.706963 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":86,"threads_started":1}
I20260619 02:29:29.707573 19140 leader_election.cc:302] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a774b506c294e4fbe76540f6ebf2f2a, a8673b38caf44739ae8e3d85aa842afb; no voters:
I20260619 02:29:29.707752 19348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8370b665876f48289ba655ce58a988ee" candidate_uuid: "a8673b38caf44739ae8e3d85aa842afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56b6c80a9651431da62ceadd08e8151b" is_pre_election: true
I20260619 02:29:29.707868 19425 raft_consensus.cc:2781] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:29:29.707924 19425 raft_consensus.cc:482] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:29:29.707968 19425 raft_consensus.cc:3037] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:29.708026 19348 raft_consensus.cc:2454] T 8370b665876f48289ba655ce58a988ee P 56b6c80a9651431da62ceadd08e8151b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8673b38caf44739ae8e3d85aa842afb in term 0.
I20260619 02:29:29.709581 19425 raft_consensus.cc:504] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } }
I20260619 02:29:29.710150 19425 leader_election.cc:288] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb [CANDIDATE]: Term 1 election: Requested vote from peers 5a774b506c294e4fbe76540f6ebf2f2a (127.17.96.193:37789), 56b6c80a9651431da62ceadd08e8151b (127.17.96.195:39809)
I20260619 02:29:29.710606 19068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8370b665876f48289ba655ce58a988ee" candidate_uuid: "a8673b38caf44739ae8e3d85aa842afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a774b506c294e4fbe76540f6ebf2f2a"
I20260619 02:29:29.710780 19068 raft_consensus.cc:3037] T 8370b665876f48289ba655ce58a988ee P 5a774b506c294e4fbe76540f6ebf2f2a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:29.710834 19348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8370b665876f48289ba655ce58a988ee" candidate_uuid: "a8673b38caf44739ae8e3d85aa842afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56b6c80a9651431da62ceadd08e8151b"
I20260619 02:29:29.710956 19348 raft_consensus.cc:3037] T 8370b665876f48289ba655ce58a988ee P 56b6c80a9651431da62ceadd08e8151b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:29.712193 19348 raft_consensus.cc:2454] T 8370b665876f48289ba655ce58a988ee P 56b6c80a9651431da62ceadd08e8151b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8673b38caf44739ae8e3d85aa842afb in term 1.
I20260619 02:29:29.712416 19068 raft_consensus.cc:2454] T 8370b665876f48289ba655ce58a988ee P 5a774b506c294e4fbe76540f6ebf2f2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8673b38caf44739ae8e3d85aa842afb in term 1.
I20260619 02:29:29.712713 19142 leader_election.cc:302] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56b6c80a9651431da62ceadd08e8151b, a8673b38caf44739ae8e3d85aa842afb; no voters:
I20260619 02:29:29.713435 19425 raft_consensus.cc:2781] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:29.713620 19425 raft_consensus.cc:686] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb [term 1 LEADER]: Becoming Leader. State: Replica: a8673b38caf44739ae8e3d85aa842afb, State: Running, Role: LEADER
I20260619 02:29:29.713930 19425 consensus_queue.cc:237] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } }
I20260619 02:29:29.716871 18910 catalog_manager.cc:5495] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb reported cstate change: term changed from 0 to 1, leader changed from <none> to a8673b38caf44739ae8e3d85aa842afb (127.17.96.194). New cstate: current_term: 1 leader_uuid: "a8673b38caf44739ae8e3d85aa842afb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } health_report { overall_health: UNKNOWN } } }
Jun 19 02:29:29 dist-test-slave-4qw6 krb5kdc[18866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836168, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:29.769567 19455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.760088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:39071 (local address 127.0.0.1:45948)
0619 02:29:29.760449 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:29.760467 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:29.760584 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:29.760892 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:29.760894 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:29.761280 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:29.761387 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.761393 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.762489 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.762491 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.763051 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.763059 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.763641 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.763643 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.763724 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.764372 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:29.764384 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:29.766088 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:29.768020 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:29.768028 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:29.768034 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:29.768276 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:29.768561 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:29.768563 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:29.768564 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:29.768641 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:29.769047 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:29.769053 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:29.769161 (+ 108us) client_negotiation.cc:769] Sending connection context
0619 02:29:29.769254 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 02:29:29.769415 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":1,"thread_start_us":170,"threads_started":1}
I20260619 02:29:29.769663 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.760173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45948 (local address 127.17.96.254:39071)
0619 02:29:29.760326 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:29.760332 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:29.760567 (+ 235us) server_negotiation.cc:407] Connection header received
0619 02:29:29.760662 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:29.760664 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:29.760724 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:29.760855 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:29.761516 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.762366 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.763173 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.763514 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.763627 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.766229 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:29.766233 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:29.766236 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:29.766261 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:29.767905 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:29.768383 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:29.768386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:29.768388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:29.768473 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:29.768723 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:29.768726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:29.768728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:29.768951 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:29.769043 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:29.769339 (+ 296us) server_negotiation.cc:299] Negotiation successful
0619 02:29:29.769483 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":59}
I20260619 02:29:29.774277 18910 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:45948:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 02:29:29.775130 18910 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:29:29.780511 19048 tablet_service.cc:1467] Processing CreateTablet for tablet 6b7fe9cd7edd41da900033ce0510f83e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6142fa9326db4684b4f49523629d0de6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:29:29.780728 19188 tablet_service.cc:1467] Processing CreateTablet for tablet 6b7fe9cd7edd41da900033ce0510f83e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6142fa9326db4684b4f49523629d0de6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:29:29.781001 19048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b7fe9cd7edd41da900033ce0510f83e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:29.781154 19188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b7fe9cd7edd41da900033ce0510f83e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:29.783725 19418 tablet_bootstrap.cc:492] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a: Bootstrap starting.
I20260619 02:29:29.784112 19328 tablet_service.cc:1467] Processing CreateTablet for tablet 6b7fe9cd7edd41da900033ce0510f83e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6142fa9326db4684b4f49523629d0de6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:29:29.784518 19328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b7fe9cd7edd41da900033ce0510f83e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:29.784802 19418 tablet_bootstrap.cc:654] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:29.784868 19421 tablet_bootstrap.cc:492] T 6b7fe9cd7edd41da900033ce0510f83e P a8673b38caf44739ae8e3d85aa842afb: Bootstrap starting.
I20260619 02:29:29.785837 19421 tablet_bootstrap.cc:654] T 6b7fe9cd7edd41da900033ce0510f83e P a8673b38caf44739ae8e3d85aa842afb: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:29.787243 19421 tablet_bootstrap.cc:492] T 6b7fe9cd7edd41da900033ce0510f83e P a8673b38caf44739ae8e3d85aa842afb: No bootstrap required, opened a new log
I20260619 02:29:29.787464 19417 tablet_bootstrap.cc:492] T 6b7fe9cd7edd41da900033ce0510f83e P 56b6c80a9651431da62ceadd08e8151b: Bootstrap starting.
I20260619 02:29:29.788295 19418 tablet_bootstrap.cc:492] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a: No bootstrap required, opened a new log
I20260619 02:29:29.788316 19417 tablet_bootstrap.cc:654] T 6b7fe9cd7edd41da900033ce0510f83e P 56b6c80a9651431da62ceadd08e8151b: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:29.788424 19418 ts_tablet_manager.cc:1397] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260619 02:29:29.788805 19421 ts_tablet_manager.cc:1397] T 6b7fe9cd7edd41da900033ce0510f83e P a8673b38caf44739ae8e3d85aa842afb: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260619 02:29:29.788938 19418 raft_consensus.cc:348] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } }
I20260619 02:29:29.789114 19418 raft_consensus.cc:374] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:29.789172 19418 raft_consensus.cc:729] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a774b506c294e4fbe76540f6ebf2f2a, State: Initialized, Role: FOLLOWER
I20260619 02:29:29.789259 19421 raft_consensus.cc:348] T 6b7fe9cd7edd41da900033ce0510f83e P a8673b38caf44739ae8e3d85aa842afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } }
I20260619 02:29:29.789374 19421 raft_consensus.cc:374] T 6b7fe9cd7edd41da900033ce0510f83e P a8673b38caf44739ae8e3d85aa842afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:29.789403 19421 raft_consensus.cc:729] T 6b7fe9cd7edd41da900033ce0510f83e P a8673b38caf44739ae8e3d85aa842afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8673b38caf44739ae8e3d85aa842afb, State: Initialized, Role: FOLLOWER
I20260619 02:29:29.789330 19418 consensus_queue.cc:260] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } }
I20260619 02:29:29.789522 19421 consensus_queue.cc:260] T 6b7fe9cd7edd41da900033ce0510f83e P a8673b38caf44739ae8e3d85aa842afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } }
I20260619 02:29:29.789697 19418 ts_tablet_manager.cc:1428] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 02:29:29.789832 19422 tablet_replica.cc:439] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } } }
I20260619 02:29:29.789987 19422 tablet_replica.cc:442] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:29:29.790592 19425 tablet_replica.cc:439] T 6b7fe9cd7edd41da900033ce0510f83e P a8673b38caf44739ae8e3d85aa842afb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } } }
I20260619 02:29:29.790782 19425 tablet_replica.cc:442] T 6b7fe9cd7edd41da900033ce0510f83e P a8673b38caf44739ae8e3d85aa842afb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:29:29.791950 19417 tablet_bootstrap.cc:492] T 6b7fe9cd7edd41da900033ce0510f83e P 56b6c80a9651431da62ceadd08e8151b: No bootstrap required, opened a new log
I20260619 02:29:29.792045 19417 ts_tablet_manager.cc:1397] T 6b7fe9cd7edd41da900033ce0510f83e P 56b6c80a9651431da62ceadd08e8151b: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260619 02:29:29.792471 19417 raft_consensus.cc:348] T 6b7fe9cd7edd41da900033ce0510f83e P 56b6c80a9651431da62ceadd08e8151b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } }
I20260619 02:29:29.792621 19417 raft_consensus.cc:374] T 6b7fe9cd7edd41da900033ce0510f83e P 56b6c80a9651431da62ceadd08e8151b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:29.792650 19417 raft_consensus.cc:729] T 6b7fe9cd7edd41da900033ce0510f83e P 56b6c80a9651431da62ceadd08e8151b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56b6c80a9651431da62ceadd08e8151b, State: Initialized, Role: FOLLOWER
I20260619 02:29:29.792783 19417 consensus_queue.cc:260] T 6b7fe9cd7edd41da900033ce0510f83e P 56b6c80a9651431da62ceadd08e8151b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } }
I20260619 02:29:29.793073 19417 ts_tablet_manager.cc:1428] T 6b7fe9cd7edd41da900033ce0510f83e P 56b6c80a9651431da62ceadd08e8151b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:29:29.793349 19435 tablet_replica.cc:439] T 6b7fe9cd7edd41da900033ce0510f83e P 56b6c80a9651431da62ceadd08e8151b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } } }
I20260619 02:29:29.793486 19435 tablet_replica.cc:442] T 6b7fe9cd7edd41da900033ce0510f83e P 56b6c80a9651431da62ceadd08e8151b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:29:29.794019 19421 ts_tablet_manager.cc:1428] T 6b7fe9cd7edd41da900033ce0510f83e P a8673b38caf44739ae8e3d85aa842afb: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20260619 02:29:29.915485 19422 raft_consensus.cc:482] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:29:29.915689 19422 raft_consensus.cc:504] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } }
I20260619 02:29:29.916920 19422 leader_election.cc:288] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8673b38caf44739ae8e3d85aa842afb (127.17.96.194:37913), 56b6c80a9651431da62ceadd08e8151b (127.17.96.195:39809)
I20260619 02:29:29.921630 19459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.917209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:39809 (local address 127.17.96.193:40119)
0619 02:29:29.917801 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:29.917815 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:29.917907 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:29.918675 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:29.918678 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:29.918695 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:29.918829 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.918835 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.919604 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.919610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.920457 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.920466 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.921170 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.921173 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.921277 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.921303 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 02:29:29.921394 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 02:29:29.921470 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":120,"threads_started":1}
I20260619 02:29:29.921617 19412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.917616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:40119 (local address 127.17.96.195:39809)
0619 02:29:29.918319 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:29.918323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:29.918338 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:29:29.918512 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:29.918514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:29.918554 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:29.918663 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:29.918944 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.919497 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.920579 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.921045 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.921175 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.921229 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:29.921425 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 02:29:29.921494 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593}
I20260619 02:29:29.921819 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.917677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:54631 (local address 127.17.96.194:37913)
0619 02:29:29.917896 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:29.917899 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:29.917910 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:29:29.917982 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:29.917986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:29.918041 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:29.918125 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:29.918657 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.919315 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.920756 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.921160 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.921315 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.921377 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:29.921586 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 02:29:29.921663 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260619 02:29:29.922048 19460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.917166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:37913 (local address 127.17.96.193:54631)
0619 02:29:29.917581 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:29.917600 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:29.917713 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:29.918176 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:29.918179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:29.918199 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:29.918488 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.918500 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.919437 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.919442 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.920573 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.920583 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.921307 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.921312 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.921431 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.921458 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 02:29:29.921565 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 02:29:29.921639 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":41,"thread_start_us":139,"threads_started":1}
I20260619 02:29:29.922205 19348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b7fe9cd7edd41da900033ce0510f83e" candidate_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56b6c80a9651431da62ceadd08e8151b" is_pre_election: true
I20260619 02:29:29.922372 19348 raft_consensus.cc:2454] T 6b7fe9cd7edd41da900033ce0510f83e P 56b6c80a9651431da62ceadd08e8151b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a774b506c294e4fbe76540f6ebf2f2a in term 0.
I20260619 02:29:29.922576 19208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b7fe9cd7edd41da900033ce0510f83e" candidate_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8673b38caf44739ae8e3d85aa842afb" is_pre_election: true
I20260619 02:29:29.922726 19208 raft_consensus.cc:2454] T 6b7fe9cd7edd41da900033ce0510f83e P a8673b38caf44739ae8e3d85aa842afb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a774b506c294e4fbe76540f6ebf2f2a in term 0.
I20260619 02:29:29.923180 19001 leader_election.cc:302] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 56b6c80a9651431da62ceadd08e8151b, 5a774b506c294e4fbe76540f6ebf2f2a, a8673b38caf44739ae8e3d85aa842afb; no voters:
I20260619 02:29:29.923475 19422 raft_consensus.cc:2781] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:29:29.923538 19422 raft_consensus.cc:482] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:29:29.923576 19422 raft_consensus.cc:3037] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:29.924646 19422 raft_consensus.cc:504] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } }
I20260619 02:29:29.925029 19422 leader_election.cc:288] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a [CANDIDATE]: Term 1 election: Requested vote from peers a8673b38caf44739ae8e3d85aa842afb (127.17.96.194:37913), 56b6c80a9651431da62ceadd08e8151b (127.17.96.195:39809)
I20260619 02:29:29.925405 19208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b7fe9cd7edd41da900033ce0510f83e" candidate_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8673b38caf44739ae8e3d85aa842afb"
I20260619 02:29:29.925426 19348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b7fe9cd7edd41da900033ce0510f83e" candidate_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56b6c80a9651431da62ceadd08e8151b"
I20260619 02:29:29.925535 19348 raft_consensus.cc:3037] T 6b7fe9cd7edd41da900033ce0510f83e P 56b6c80a9651431da62ceadd08e8151b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:29.925546 19208 raft_consensus.cc:3037] T 6b7fe9cd7edd41da900033ce0510f83e P a8673b38caf44739ae8e3d85aa842afb [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:29.926559 19208 raft_consensus.cc:2454] T 6b7fe9cd7edd41da900033ce0510f83e P a8673b38caf44739ae8e3d85aa842afb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a774b506c294e4fbe76540f6ebf2f2a in term 1.
I20260619 02:29:29.926563 19348 raft_consensus.cc:2454] T 6b7fe9cd7edd41da900033ce0510f83e P 56b6c80a9651431da62ceadd08e8151b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a774b506c294e4fbe76540f6ebf2f2a in term 1.
I20260619 02:29:29.926882 19001 leader_election.cc:302] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a774b506c294e4fbe76540f6ebf2f2a, a8673b38caf44739ae8e3d85aa842afb; no voters:
I20260619 02:29:29.927083 19422 raft_consensus.cc:2781] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:29.927289 19422 raft_consensus.cc:686] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a [term 1 LEADER]: Becoming Leader. State: Replica: 5a774b506c294e4fbe76540f6ebf2f2a, State: Running, Role: LEADER
I20260619 02:29:29.927505 19422 consensus_queue.cc:237] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } }
I20260619 02:29:29.928653 19461 tablet_replica.cc:439] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a: TxnStatusTablet state changed. Reason: New leader 5a774b506c294e4fbe76540f6ebf2f2a. Latest consensus state: current_term: 1 leader_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } } }
I20260619 02:29:29.928782 19461 tablet_replica.cc:442] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:29:29.929067 19463 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 02:29:29.929236 19463 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 02:29:29.929936 18910 catalog_manager.cc:5495] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a774b506c294e4fbe76540f6ebf2f2a (127.17.96.193). New cstate: current_term: 1 leader_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } health_report { overall_health: UNKNOWN } } }
I20260619 02:29:29.957587 19415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.953457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39164 (local address 127.17.96.193:37789)
0619 02:29:29.953584 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:29.953587 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:29.953697 (+ 110us) server_negotiation.cc:407] Connection header received
0619 02:29:29.953916 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:29.953919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:29.953976 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:29.954059 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:29:29.954332 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.954835 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.956107 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.956325 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.956404 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.956744 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:29:29.957195 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:29:29.957279 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:29.957401 (+ 122us) server_negotiation.cc:299] Negotiation successful
0619 02:29:29.957455 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 02:29:29.957638 19455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.953358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:37789 (local address 127.0.0.1:39164)
0619 02:29:29.953621 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:29.953637 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:29.953787 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:29.954100 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:29.954103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:29.954112 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:29:29.954228 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.954234 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.954961 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.954964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.955989 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.956001 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.956437 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.956440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.956542 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.956602 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:29:29.957291 (+ 689us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:29:29.957307 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 02:29:29.957383 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 02:29:29.957483 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161}
I20260619 02:29:29.960660 19208 raft_consensus.cc:1261] T 6b7fe9cd7edd41da900033ce0510f83e P a8673b38caf44739ae8e3d85aa842afb [term 1 FOLLOWER]: Refusing update from remote peer 5a774b506c294e4fbe76540f6ebf2f2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:29:29.960894 19348 raft_consensus.cc:1261] T 6b7fe9cd7edd41da900033ce0510f83e P 56b6c80a9651431da62ceadd08e8151b [term 1 FOLLOWER]: Refusing update from remote peer 5a774b506c294e4fbe76540f6ebf2f2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:29:29.961199 19422 consensus_queue.cc:1035] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:29:29.961422 19461 consensus_queue.cc:1035] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:29:29.963106 19435 tablet_replica.cc:439] T 6b7fe9cd7edd41da900033ce0510f83e P 56b6c80a9651431da62ceadd08e8151b: TxnStatusTablet state changed. Reason: New leader 5a774b506c294e4fbe76540f6ebf2f2a. Latest consensus state: current_term: 1 leader_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } } }
I20260619 02:29:29.963236 19435 tablet_replica.cc:442] T 6b7fe9cd7edd41da900033ce0510f83e P 56b6c80a9651431da62ceadd08e8151b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:29:29.963994 19425 tablet_replica.cc:439] T 6b7fe9cd7edd41da900033ce0510f83e P a8673b38caf44739ae8e3d85aa842afb: TxnStatusTablet state changed. Reason: New leader 5a774b506c294e4fbe76540f6ebf2f2a. Latest consensus state: current_term: 1 leader_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } } }
I20260619 02:29:29.964154 19425 tablet_replica.cc:442] T 6b7fe9cd7edd41da900033ce0510f83e P a8673b38caf44739ae8e3d85aa842afb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:29:29.965615 19431 mvcc.cc:205] Tried to move back new op lower bound from 7298400952153624576 to 7298400952025530368. Current Snapshot: MvccSnapshot[applied={T|T < 7298400952153624576}]
I20260619 02:29:29.965739 19422 tablet_replica.cc:439] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } } }
I20260619 02:29:29.965891 19422 tablet_replica.cc:442] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:29:29.967217 19425 tablet_replica.cc:439] T 6b7fe9cd7edd41da900033ce0510f83e P a8673b38caf44739ae8e3d85aa842afb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } } }
I20260619 02:29:29.967335 19425 tablet_replica.cc:442] T 6b7fe9cd7edd41da900033ce0510f83e P a8673b38caf44739ae8e3d85aa842afb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:29:29.970850 19461 tablet_replica.cc:439] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } } }
I20260619 02:29:29.971016 19461 tablet_replica.cc:442] T 6b7fe9cd7edd41da900033ce0510f83e P 5a774b506c294e4fbe76540f6ebf2f2a: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:29:29.971572 19430 mvcc.cc:205] Tried to move back new op lower bound from 7298400952153624576 to 7298400952025530368. Current Snapshot: MvccSnapshot[applied={T|T < 7298400952153624576}]
I20260619 02:29:29.971714 19435 tablet_replica.cc:439] T 6b7fe9cd7edd41da900033ce0510f83e P 56b6c80a9651431da62ceadd08e8151b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 } } peers { permanent_uuid: "a8673b38caf44739ae8e3d85aa842afb" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37913 } } peers { permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 } } }
I20260619 02:29:29.971860 19435 tablet_replica.cc:442] T 6b7fe9cd7edd41da900033ce0510f83e P 56b6c80a9651431da62ceadd08e8151b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:29:29.978111 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.974218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35830 (local address 127.17.96.194:37913)
0619 02:29:29.974339 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:29.974344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:29.974363 (+ 19us) server_negotiation.cc:407] Connection header received
0619 02:29:29.974534 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:29.974537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:29.974591 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:29.974678 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:29:29.974924 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.975418 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.976574 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.976793 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.976866 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.977153 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:29:29.977650 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:29:29.977824 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:29.977884 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 02:29:29.977937 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 02:29:29.989508 19415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.985976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:37905 (local address 127.17.96.193:37789)
0619 02:29:29.986104 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:29.986108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:29.986318 (+ 210us) server_negotiation.cc:407] Connection header received
0619 02:29:29.986484 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:29.986487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:29.986543 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:29.986625 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:29:29.986905 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.987430 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.988378 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.988598 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.988708 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.989029 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:29:29.989105 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:29:29.989183 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:29.989295 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 02:29:29.989349 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 02:29:29.989840 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:29.985803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:37789 (local address 127.17.96.194:37905)
0619 02:29:29.986223 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:29.986235 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:29.986373 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:29.986647 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:29.986650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:29.986658 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:29:29.986795 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.986801 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.987568 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.987573 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.988243 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:29.988253 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:29.988712 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:29.988716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:29.988833 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:29.988880 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:29:29.989187 (+ 307us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:29:29.989214 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 02:29:29.989281 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 02:29:29.989366 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":108,"threads_started":1}
I20260619 02:29:29.997735 19348 raft_consensus.cc:1261] T 8370b665876f48289ba655ce58a988ee P 56b6c80a9651431da62ceadd08e8151b [term 1 FOLLOWER]: Refusing update from remote peer a8673b38caf44739ae8e3d85aa842afb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:29:29.997963 19068 raft_consensus.cc:1261] T 8370b665876f48289ba655ce58a988ee P 5a774b506c294e4fbe76540f6ebf2f2a [term 1 FOLLOWER]: Refusing update from remote peer a8673b38caf44739ae8e3d85aa842afb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:29:29.998323 19425 consensus_queue.cc:1035] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb [LEADER]: Connected to new peer: Peer: permanent_uuid: "56b6c80a9651431da62ceadd08e8151b" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:29:29.998549 19448 consensus_queue.cc:1035] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a774b506c294e4fbe76540f6ebf2f2a" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:29:30.003265 19432 mvcc.cc:205] Tried to move back new op lower bound from 7298400952306106368 to 7298400951152574464. Current Snapshot: MvccSnapshot[applied={T|T < 7298400952306106368}]
I20260619 02:29:30.023747 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:30.019719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:42517 (local address 127.17.96.194:37913)
0619 02:29:30.019854 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:30.019860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:30.020128 (+ 268us) server_negotiation.cc:407] Connection header received
0619 02:29:30.020328 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:30.020334 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:30.020385 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:30.020435 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:29:30.020873 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:30.021570 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:30.022606 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:30.022849 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:30.022965 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:30.023253 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:29:30.023343 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:29:30.023421 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:30.023514 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 02:29:30.023568 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 02:29:30.023757 19476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:30.019634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:37913 (local address 127.17.96.193:42517)
0619 02:29:30.020038 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:30.020051 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:30.020194 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:30.020551 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:30.020553 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:30.020559 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:29:30.020711 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:30.020718 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:30.021702 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:30.021708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:30.022474 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:30.022484 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:30.022962 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:30.022966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:30.023071 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:30.023116 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:29:30.023436 (+ 320us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:29:30.023444 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 02:29:30.023505 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 02:29:30.023590 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":109,"threads_started":1}
I20260619 02:29:30.049798 18909 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:29:30.049988 18909 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:29:30.052898 18909 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab5a9f80d85747c0ae1a978733444848: Sending DeleteTablet for 3 replicas of tablet 8370b665876f48289ba655ce58a988ee
I20260619 02:29:30.053857 19048 tablet_service.cc:1514] Processing DeleteTablet for tablet 8370b665876f48289ba655ce58a988ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:30 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:39154
I20260619 02:29:30.054062 19442 tablet_replica.cc:330] T 8370b665876f48289ba655ce58a988ee P 5a774b506c294e4fbe76540f6ebf2f2a: stopping tablet replica
I20260619 02:29:30.054000 19328 tablet_service.cc:1514] Processing DeleteTablet for tablet 8370b665876f48289ba655ce58a988ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:30 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:46180
I20260619 02:29:30.054176 19442 raft_consensus.cc:2229] T 8370b665876f48289ba655ce58a988ee P 5a774b506c294e4fbe76540f6ebf2f2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:29:30.054284 19440 tablet_replica.cc:330] T 8370b665876f48289ba655ce58a988ee P 56b6c80a9651431da62ceadd08e8151b: stopping tablet replica
I20260619 02:29:30.054324 19442 raft_consensus.cc:2258] T 8370b665876f48289ba655ce58a988ee P 5a774b506c294e4fbe76540f6ebf2f2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:30.054405 19440 raft_consensus.cc:2229] T 8370b665876f48289ba655ce58a988ee P 56b6c80a9651431da62ceadd08e8151b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:29:30.054536 19440 raft_consensus.cc:2258] T 8370b665876f48289ba655ce58a988ee P 56b6c80a9651431da62ceadd08e8151b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:30.055402 19440 ts_tablet_manager.cc:1905] T 8370b665876f48289ba655ce58a988ee P 56b6c80a9651431da62ceadd08e8151b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:30.055477 19442 ts_tablet_manager.cc:1905] T 8370b665876f48289ba655ce58a988ee P 5a774b506c294e4fbe76540f6ebf2f2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:30.056280 19188 tablet_service.cc:1514] Processing DeleteTablet for tablet 8370b665876f48289ba655ce58a988ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:30 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:35818
I20260619 02:29:30.057413 19441 tablet_replica.cc:330] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb: stopping tablet replica
I20260619 02:29:30.057547 19441 raft_consensus.cc:2229] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:29:30.057958 19441 raft_consensus.cc:2258] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:30.058849 19440 ts_tablet_manager.cc:1918] T 8370b665876f48289ba655ce58a988ee P 56b6c80a9651431da62ceadd08e8151b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:29:30.058882 19441 ts_tablet_manager.cc:1905] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:30.058947 19440 log.cc:1196] T 8370b665876f48289ba655ce58a988ee P 56b6c80a9651431da62ceadd08e8151b: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/8370b665876f48289ba655ce58a988ee
I20260619 02:29:30.059253 19440 ts_tablet_manager.cc:1939] T 8370b665876f48289ba655ce58a988ee P 56b6c80a9651431da62ceadd08e8151b: Deleting consensus metadata
I20260619 02:29:30.060473 18898 catalog_manager.cc:4841] TS 56b6c80a9651431da62ceadd08e8151b (127.17.96.195:39809): tablet 8370b665876f48289ba655ce58a988ee (table test-table [id=be1ab3de38334045b8273a4cdd78621d]) successfully deleted
I20260619 02:29:30.061997 19441 ts_tablet_manager.cc:1918] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:29:30.062091 19441 log.cc:1196] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/8370b665876f48289ba655ce58a988ee
I20260619 02:29:30.062353 19441 ts_tablet_manager.cc:1939] T 8370b665876f48289ba655ce58a988ee P a8673b38caf44739ae8e3d85aa842afb: Deleting consensus metadata
I20260619 02:29:30.063171 18897 catalog_manager.cc:4841] TS a8673b38caf44739ae8e3d85aa842afb (127.17.96.194:37913): tablet 8370b665876f48289ba655ce58a988ee (table test-table [id=be1ab3de38334045b8273a4cdd78621d]) successfully deleted
I20260619 02:29:30.064430 19442 ts_tablet_manager.cc:1918] T 8370b665876f48289ba655ce58a988ee P 5a774b506c294e4fbe76540f6ebf2f2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:29:30.064517 19442 log.cc:1196] T 8370b665876f48289ba655ce58a988ee P 5a774b506c294e4fbe76540f6ebf2f2a: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781836162001484-17795-0/minicluster-data/ts-0/wal/wals/8370b665876f48289ba655ce58a988ee
I20260619 02:29:30.064797 19442 ts_tablet_manager.cc:1939] T 8370b665876f48289ba655ce58a988ee P 5a774b506c294e4fbe76540f6ebf2f2a: Deleting consensus metadata
I20260619 02:29:30.065568 18896 catalog_manager.cc:4841] TS 5a774b506c294e4fbe76540f6ebf2f2a (127.17.96.193:37789): tablet 8370b665876f48289ba655ce58a988ee (table test-table [id=be1ab3de38334045b8273a4cdd78621d]) successfully deleted
Jun 19 02:29:30 dist-test-slave-4qw6 krb5kdc[18866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836169, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.96.193@KRBTEST.COM
I20260619 02:29:30.076385 19415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:30.056772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39176 (local address 127.17.96.193:37789)
0619 02:29:30.056897 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:30.056902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:30.062332 (+ 5430us) server_negotiation.cc:407] Connection header received
0619 02:29:30.062646 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:30.062650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:30.062710 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:30.062776 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:30.063943 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:30.064758 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:30.066497 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:30.066867 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:30.066988 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:30.071272 (+ 4284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:30.071287 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:30.071306 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:30.071342 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:30.073636 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:30.074256 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:30.074264 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:30.074265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:30.074401 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:30.074746 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:30.074750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:30.074752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:30.075107 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:30.075906 (+ 799us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:30.076009 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 02:29:30.076146 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260619 02:29:30.076915 19108 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39176
I20260619 02:29:30.092010 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:30.079436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45950 (local address 127.17.96.254:39071)
0619 02:29:30.079577 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:30.079582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:30.082172 (+ 2590us) server_negotiation.cc:407] Connection header received
0619 02:29:30.082317 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:30.082321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:30.082384 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:30.082481 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:30.083410 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:30.084250 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:30.085468 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:30.085896 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:30.086001 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:30.087864 (+ 1863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:30.087868 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:30.087871 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:30.087905 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:30.089849 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:30.090376 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:30.090380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:30.090382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:30.090496 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:30.090931 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:30.090935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:30.090937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:30.091235 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:30.091388 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:30.091674 (+ 286us) server_negotiation.cc:299] Negotiation successful
0619 02:29:30.091831 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260619 02:29:30.092473 18909 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45950
I20260619 02:29:30.094172 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 18983
I20260619 02:29:30.101943 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 19123
I20260619 02:29:30.108352 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 19263
I20260619 02:29:30.115641 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 18879
2026-06-19T02:29:30Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2885 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:29:30 dist-test-slave-4qw6 krb5kdc[19485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:29:30 dist-test-slave-4qw6 krb5kdc[19485](info): set up 2 sockets
Jun 19 02:29:30 dist-test-slave-4qw6 krb5kdc[19485](info): commencing operation
krb5kdc: starting...
W20260619 02:29:31.214089 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:29:31 dist-test-slave-4qw6 krb5kdc[19485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:29:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:29:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:29:31.362432 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:42351
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:34507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:42351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:31.498422 19498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:31.498849 19498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:31.498963 19498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:31.504750 19498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:29:31.504944 19498 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 02:29:31.504983 19498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:31.505029 19498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:31.505080 19498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:29:31.505127 19498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:29:31.514472 19498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:34507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:42351
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:42351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:31.516274 19498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:31.518147 19498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:29:31.529325 19498 server_base.cc:1034] running on GCE node
W20260619 02:29:31.529409 19504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:31.529275 19503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:31.529285 19506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:31.530297 19498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:31.531667 19498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:31.532857 19498 hybrid_clock.cc:648] HybridClock initialized: now 1781836171532824 us; error 56 us; skew 500 ppm
Jun 19 02:29:31 dist-test-slave-4qw6 krb5kdc[19485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836171, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:31.536284 19498 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:29:31.537348 19498 webserver.cc:460] Webserver started at http://127.17.96.254:44831/ using document root <none> and password file <none>
I20260619 02:29:31.538092 19498 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:31.538170 19498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:31.538367 19498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:31.540784 19498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "9f20885720a046c2898e54b1f82ed714"
format_stamp: "Formatted at 2026-06-19 02:29:31 on dist-test-slave-4qw6"
server_key: "42ce981ac82481df2d02aeca2895d92b"
server_key_iv: "c657d7b7bdb882b42df7f71a87b9f33c"
server_key_version: "encryptionkey@0"
I20260619 02:29:31.541389 19498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "9f20885720a046c2898e54b1f82ed714"
format_stamp: "Formatted at 2026-06-19 02:29:31 on dist-test-slave-4qw6"
server_key: "42ce981ac82481df2d02aeca2895d92b"
server_key_iv: "c657d7b7bdb882b42df7f71a87b9f33c"
server_key_version: "encryptionkey@0"
I20260619 02:29:31.546075 19498 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260619 02:29:31.549001 19513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:31.550577 19498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:29:31.550746 19498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "9f20885720a046c2898e54b1f82ed714"
format_stamp: "Formatted at 2026-06-19 02:29:31 on dist-test-slave-4qw6"
server_key: "42ce981ac82481df2d02aeca2895d92b"
server_key_iv: "c657d7b7bdb882b42df7f71a87b9f33c"
server_key_version: "encryptionkey@0"
I20260619 02:29:31.550904 19498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:31.566331 19498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:31.570010 19498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:31.570253 19498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:31.580484 19498 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:42351
I20260619 02:29:31.580554 19575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:42351 every 8 connection(s)
I20260619 02:29:31.581816 19498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:29:31.585592 19576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:31.591858 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 19498
I20260619 02:29:31.592002 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:29:31.592310 17795 external_mini_cluster.cc:1428] Setting key 68e4b230e20eabf5072884e002bff301
I20260619 02:29:31.592432 19576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714: Bootstrap starting.
I20260619 02:29:31.595300 19576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:31.596131 19576 log.cc:824] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:31.598971 19576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714: No bootstrap required, opened a new log
Jun 19 02:29:31 dist-test-slave-4qw6 krb5kdc[19485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:31.604748 19576 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f20885720a046c2898e54b1f82ed714" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42351 } }
I20260619 02:29:31.605145 19576 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:31.605242 19576 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f20885720a046c2898e54b1f82ed714, State: Initialized, Role: FOLLOWER
I20260619 02:29:31.605911 19576 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f20885720a046c2898e54b1f82ed714" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42351 } }
I20260619 02:29:31.606112 19576 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:29:31.606220 19576 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:29:31.606355 19576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:31.607828 19576 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f20885720a046c2898e54b1f82ed714" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42351 } }
I20260619 02:29:31.608440 19576 leader_election.cc:302] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f20885720a046c2898e54b1f82ed714; no voters:
I20260619 02:29:31.609009 19576 leader_election.cc:288] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:29:31.609200 19581 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:31.609599 19581 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714 [term 1 LEADER]: Becoming Leader. State: Replica: 9f20885720a046c2898e54b1f82ed714, State: Running, Role: LEADER
I20260619 02:29:31.610173 19581 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f20885720a046c2898e54b1f82ed714" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42351 } }
I20260619 02:29:31.610767 19576 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:29:31.611167 19579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:31.593987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48230 (local address 127.17.96.254:42351)
0619 02:29:31.594506 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:31.594524 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:31.594563 (+ 39us) server_negotiation.cc:407] Connection header received
0619 02:29:31.595287 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:31.595307 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:31.595751 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:31.596151 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:31.597042 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:31.598151 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:31.599110 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:31.599738 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:31.599951 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:31.603113 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:31.603120 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:31.603140 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:31.603183 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:31.606169 (+ 2986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:31.606969 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:31.606976 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:31.606983 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:31.607147 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:31.607524 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:31.607534 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:31.607536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:31.608181 (+ 645us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:31.608525 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:31.609252 (+ 727us) server_negotiation.cc:299] Negotiation successful
0619 02:29:31.609970 (+ 718us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":173,"threads_started":1}
I20260619 02:29:31.612641 19582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f20885720a046c2898e54b1f82ed714" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f20885720a046c2898e54b1f82ed714" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42351 } } }
I20260619 02:29:31.612852 19582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:31.612757 19583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f20885720a046c2898e54b1f82ed714. Latest consensus state: current_term: 1 leader_uuid: "9f20885720a046c2898e54b1f82ed714" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f20885720a046c2898e54b1f82ed714" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42351 } } }
I20260619 02:29:31.613835 19583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714 [sys.catalog]: This master's current role is: LEADER
W20260619 02:29:31.616986 19596 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:29:31.617115 19596 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:29:31.617244 19593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:29:31.618355 19593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:29:31.626467 19593 catalog_manager.cc:1347] Generated new cluster ID: f6e37bf2469f40eb8483f23d64060286
I20260619 02:29:31.626602 19593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:29:31.639955 19593 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:29:31.641191 19593 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:29:31.645867 19593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714: Generated new TSK 0
I20260619 02:29:31.646795 19593 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.96.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
I20260619 02:29:31.690034 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:42351
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:34507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:31.829159 19602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:31.829499 19602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:31.829555 19602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:31.834432 19602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:31.834532 19602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:31.834579 19602 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 02:29:31.834600 19602 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:29:31.834640 19602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:29:31.841060 19602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:34507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.96.254:42351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:31.842499 19602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:31.843698 19602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:31.855389 19607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:31.855332 19608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:31.855492 19602 server_base.cc:1034] running on GCE node
W20260619 02:29:31.855337 19610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:31.856253 19602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:31.857072 19602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:31.860407 19602 hybrid_clock.cc:648] HybridClock initialized: now 1781836171860326 us; error 114 us; skew 500 ppm
Jun 19 02:29:31 dist-test-slave-4qw6 krb5kdc[19485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836171, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:31.864450 19602 init.cc:377] Logged in from keytab as kudu/127.17.96.193@KRBTEST.COM (short username kudu)
I20260619 02:29:31.865592 19602 webserver.cc:460] Webserver started at http://127.17.96.193:46819/ using document root <none> and password file <none>
I20260619 02:29:31.866359 19602 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:31.866446 19602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:31.866639 19602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:31.869572 19602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "cffd4b4f2c73451196a44fcc9faf826f"
format_stamp: "Formatted at 2026-06-19 02:29:31 on dist-test-slave-4qw6"
server_key: "e2211f3f9d0a2b77abed5bc3a0795a57"
server_key_iv: "61b2a23248e0562cf060153d0adcc2ad"
server_key_version: "encryptionkey@0"
I20260619 02:29:31.870602 19602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "cffd4b4f2c73451196a44fcc9faf826f"
format_stamp: "Formatted at 2026-06-19 02:29:31 on dist-test-slave-4qw6"
server_key: "e2211f3f9d0a2b77abed5bc3a0795a57"
server_key_iv: "61b2a23248e0562cf060153d0adcc2ad"
server_key_version: "encryptionkey@0"
I20260619 02:29:31.876641 19602 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260619 02:29:31.880077 19617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:31.881940 19602 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:29:31.882120 19602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "cffd4b4f2c73451196a44fcc9faf826f"
format_stamp: "Formatted at 2026-06-19 02:29:31 on dist-test-slave-4qw6"
server_key: "e2211f3f9d0a2b77abed5bc3a0795a57"
server_key_iv: "61b2a23248e0562cf060153d0adcc2ad"
server_key_version: "encryptionkey@0"
I20260619 02:29:31.882290 19602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:31.909921 19602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:31.914485 19602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:31.914754 19602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:31.916554 19602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:31.916659 19602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:31.916746 19602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:31.916800 19602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:31.931277 19602 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:45915
Jun 19 02:29:31 dist-test-slave-4qw6 krb5kdc[19485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836171, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:31.931368 19732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:45915 every 8 connection(s)
I20260619 02:29:31.932572 19602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:29:31.936561 19579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:31.917900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:57737 (local address 127.17.96.254:42351)
0619 02:29:31.918058 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:31.918062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:31.920383 (+ 2321us) server_negotiation.cc:407] Connection header received
0619 02:29:31.922310 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:31.922314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:31.922393 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:31.922548 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:31.924820 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:31.925881 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:31.927612 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:31.928045 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:31.928235 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:31.932247 (+ 4012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:31.932251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:31.932253 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:31.932285 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:31.934204 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:31.934775 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:31.934779 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:31.934780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:31.934867 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:31.935214 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:31.935217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:31.935218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:31.935422 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:31.935505 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:31.936186 (+ 681us) server_negotiation.cc:299] Negotiation successful
0619 02:29:31.936381 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 02:29:31.937340 19626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:31.919011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:42351 (local address 127.17.96.193:57737)
0619 02:29:31.920094 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:31.920211 (+ 117us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:31.921879 (+ 1668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:31.922914 (+ 1035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:31.922927 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:31.923577 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:31.924557 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:31.924599 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:31.926115 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:31.926119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:31.927452 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:31.927464 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:31.928357 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:31.928362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:31.928613 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:31.929669 (+ 1056us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:31.929701 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:31.931939 (+ 2238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:31.934336 (+ 2397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:31.934341 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:31.934354 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:31.934663 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:31.934957 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:31.934959 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:31.934961 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:31.935125 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:31.935517 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:31.935523 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:31.935804 (+ 281us) client_negotiation.cc:769] Sending connection context
0619 02:29:31.936002 (+ 198us) client_negotiation.cc:240] Negotiation successful
0619 02:29:31.936310 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"thread_start_us":366,"threads_started":1}
I20260619 02:29:31.939481 19733 heartbeater.cc:347] Connected to a master server at 127.17.96.254:42351
I20260619 02:29:31.939925 19733 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:31.939961 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 19602
I20260619 02:29:31.940106 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:29:31.940433 17795 external_mini_cluster.cc:1428] Setting key c80b3515b720015d81c771e98a53707d
I20260619 02:29:31.940737 19733 heartbeater.cc:511] Master 127.17.96.254:42351 requested a full tablet report, sending...
I20260619 02:29:31.943212 19530 ts_manager.cc:194] Registered new tserver with Master: cffd4b4f2c73451196a44fcc9faf826f (127.17.96.193:45915)
I20260619 02:29:31.945385 19530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:57737
WARNING: no policy specified for kudu/127.17.96.194@KRBTEST.COM; defaulting to no policy
I20260619 02:29:31.954936 19739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:31.943043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:42351 (local address 127.17.96.193:46705)
0619 02:29:31.945356 (+ 2313us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:31.945376 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:31.945553 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:31.946017 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:31.946020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:31.946309 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:31.946457 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:31.946464 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:31.947717 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:31.947724 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:31.948890 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:31.948907 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:31.949659 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:31.949668 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:31.949872 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:31.950613 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:31.950636 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:31.951181 (+ 545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:31.953211 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:31.953218 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:31.953224 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:31.953463 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:31.953729 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:31.953731 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:31.953732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:31.953886 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:31.954420 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:31.954423 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:31.954522 (+ 99us) client_negotiation.cc:769] Sending connection context
0619 02:29:31.954622 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 02:29:31.954760 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2211,"mutex_wait_us":3,"thread_start_us":129,"threads_started":1}
I20260619 02:29:31.954936 19579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:31.943150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:46705 (local address 127.17.96.254:42351)
0619 02:29:31.945145 (+ 1995us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:31.945151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:31.945519 (+ 368us) server_negotiation.cc:407] Connection header received
0619 02:29:31.945699 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:31.945702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:31.945787 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:31.945849 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:31.946614 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:31.947535 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:31.949154 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:31.949446 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:31.949591 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:31.951395 (+ 1804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:31.951399 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:31.951401 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:31.951431 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:31.953081 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:31.953563 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:31.953566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:31.953568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:31.953646 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:31.954015 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:31.954018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:31.954020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:31.954260 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:31.954417 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:31.954672 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 02:29:31.954789 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1885}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
I20260619 02:29:31.973640 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:42351
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:34507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:32.113651 19742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:32.114043 19742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:32.114101 19742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:32.118609 19742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:32.118692 19742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:32.118737 19742 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 02:29:32.118754 19742 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:29:32.118790 19742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:29:32.124948 19742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:34507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.96.254:42351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:32.126405 19742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:32.127563 19742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:32.139494 19747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:32.139518 19750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:32.139518 19748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:32.139820 19742 server_base.cc:1034] running on GCE node
I20260619 02:29:32.140699 19742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:32.141878 19742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:32.143122 19742 hybrid_clock.cc:648] HybridClock initialized: now 1781836172143109 us; error 54 us; skew 500 ppm
Jun 19 02:29:32 dist-test-slave-4qw6 krb5kdc[19485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836172, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:32.151124 19742 init.cc:377] Logged in from keytab as kudu/127.17.96.194@KRBTEST.COM (short username kudu)
I20260619 02:29:32.152223 19742 webserver.cc:460] Webserver started at http://127.17.96.194:40677/ using document root <none> and password file <none>
I20260619 02:29:32.152905 19742 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:32.153012 19742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:32.153250 19742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:32.155968 19742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "8223df6185ea409ebbdd26b192a64500"
format_stamp: "Formatted at 2026-06-19 02:29:32 on dist-test-slave-4qw6"
server_key: "275a5e1f4d9d0efa752724570396ce6e"
server_key_iv: "ee09614fe08a6cc43ef5321ef4adccb8"
server_key_version: "encryptionkey@0"
I20260619 02:29:32.156652 19742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "8223df6185ea409ebbdd26b192a64500"
format_stamp: "Formatted at 2026-06-19 02:29:32 on dist-test-slave-4qw6"
server_key: "275a5e1f4d9d0efa752724570396ce6e"
server_key_iv: "ee09614fe08a6cc43ef5321ef4adccb8"
server_key_version: "encryptionkey@0"
I20260619 02:29:32.162860 19742 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.006s
I20260619 02:29:32.166257 19757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:32.167985 19742 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260619 02:29:32.168206 19742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "8223df6185ea409ebbdd26b192a64500"
format_stamp: "Formatted at 2026-06-19 02:29:32 on dist-test-slave-4qw6"
server_key: "275a5e1f4d9d0efa752724570396ce6e"
server_key_iv: "ee09614fe08a6cc43ef5321ef4adccb8"
server_key_version: "encryptionkey@0"
I20260619 02:29:32.168372 19742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:32.181313 19742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:32.185341 19742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:32.185603 19742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:32.187510 19742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:32.187611 19742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:32.187716 19742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:32.187754 19742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:32.199939 19742 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:41639
I20260619 02:29:32.199985 19872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:41639 every 8 connection(s)
Jun 19 02:29:32 dist-test-slave-4qw6 krb5kdc[19485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836172, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:32.201130 19742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:29:32.202163 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 19742
I20260619 02:29:32.202325 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:29:32.202765 17795 external_mini_cluster.cc:1428] Setting key 0d70743567b724d05f0d0e7d29bce444
I20260619 02:29:32.205999 19579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:32.189184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:33545 (local address 127.17.96.254:42351)
0619 02:29:32.189353 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:32.189357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:32.190739 (+ 1382us) server_negotiation.cc:407] Connection header received
0619 02:29:32.192407 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:32.192414 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:32.192500 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:32.192623 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:32.194499 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.195523 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.196951 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.197402 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.197553 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:32.201093 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:32.201100 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:32.201107 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:32.201147 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:32.203221 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:32.203931 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:32.203935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:32.203937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:32.204037 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:32.204506 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:32.204509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:32.204510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:32.204724 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:32.204808 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:32.205615 (+ 807us) server_negotiation.cc:299] Negotiation successful
0619 02:29:32.205838 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 02:29:32.207123 19767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:32.189849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:42351 (local address 127.17.96.194:33545)
0619 02:29:32.190512 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:32.190607 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:32.192009 (+ 1402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:32.192890 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:32.192903 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:32.193555 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:32.194318 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:32.194340 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.195702 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.195705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:32.196780 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:32.196793 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.197635 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.197641 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:32.197951 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:32.198789 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:32.198811 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:32.200855 (+ 2044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:32.203404 (+ 2549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:32.203411 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:32.203429 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:32.203793 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:32.204165 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:32.204169 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:32.204174 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:32.204365 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:32.204848 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:32.204856 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:32.205241 (+ 385us) client_negotiation.cc:769] Sending connection context
0619 02:29:32.205485 (+ 244us) client_negotiation.cc:240] Negotiation successful
0619 02:29:32.205976 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":159,"threads_started":1}
I20260619 02:29:32.208642 19873 heartbeater.cc:347] Connected to a master server at 127.17.96.254:42351
I20260619 02:29:32.208806 19873 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:32.209482 19873 heartbeater.cc:511] Master 127.17.96.254:42351 requested a full tablet report, sending...
I20260619 02:29:32.211175 19525 ts_manager.cc:194] Registered new tserver with Master: 8223df6185ea409ebbdd26b192a64500 (127.17.96.194:41639)
I20260619 02:29:32.211992 19525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:33545
WARNING: no policy specified for kudu/127.17.96.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.195@KRBTEST.COM" created.
I20260619 02:29:32.222198 19579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:32.212404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:55157 (local address 127.17.96.254:42351)
0619 02:29:32.212554 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:32.212563 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:32.212575 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:29:32.212631 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:32.212634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:32.212676 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:32.212780 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:32.213381 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.214212 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.215396 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.215665 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.215795 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:32.217459 (+ 1664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:32.217465 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:32.217474 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:32.217505 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:32.219585 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:32.220408 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:32.220413 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:32.220415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:32.220591 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:32.221000 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:32.221008 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:32.221010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:32.221304 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:32.221458 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:32.221834 (+ 376us) server_negotiation.cc:299] Negotiation successful
0619 02:29:32.221978 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 02:29:32.222218 19880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:32.211048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:42351 (local address 127.17.96.194:55157)
0619 02:29:32.211421 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:32.211439 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:32.211543 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:32.212799 (+ 1256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:32.212802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:32.213066 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:32.213194 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:32.213202 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.214417 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.214421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:32.215246 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:32.215256 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.215813 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.215818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:32.215925 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:32.216638 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:32.216658 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:32.217135 (+ 477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:32.219865 (+ 2730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:32.219869 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:32.219871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:32.220242 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:32.220725 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:32.220731 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:32.220734 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:32.220874 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:32.221457 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:32.221460 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:32.221577 (+ 117us) client_negotiation.cc:769] Sending connection context
0619 02:29:32.221719 (+ 142us) client_negotiation.cc:240] Negotiation successful
0619 02:29:32.221973 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":148,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
I20260619 02:29:32.242115 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:42351
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:34507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:32.382397 19882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:32.382745 19882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:32.382871 19882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:32.387645 19882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:32.387740 19882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:32.387831 19882 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 02:29:32.387873 19882 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:29:32.388002 19882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:29:32.394565 19882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:34507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.96.254:42351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:32.395941 19882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:32.397174 19882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:32.411005 19888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:32.410993 19887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:32.410993 19890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:32.411760 19882 server_base.cc:1034] running on GCE node
I20260619 02:29:32.413067 19882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:32.414098 19882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:32.415318 19882 hybrid_clock.cc:648] HybridClock initialized: now 1781836172415273 us; error 64 us; skew 500 ppm
Jun 19 02:29:32 dist-test-slave-4qw6 krb5kdc[19485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836172, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:32.419004 19882 init.cc:377] Logged in from keytab as kudu/127.17.96.195@KRBTEST.COM (short username kudu)
I20260619 02:29:32.420212 19882 webserver.cc:460] Webserver started at http://127.17.96.195:45931/ using document root <none> and password file <none>
I20260619 02:29:32.420919 19882 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:32.421005 19882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:32.421268 19882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:32.423834 19882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "5657d994279149cfac5b1b909f4f5b63"
format_stamp: "Formatted at 2026-06-19 02:29:32 on dist-test-slave-4qw6"
server_key: "d25b72e9beb0d4ac4be919fb89724de9"
server_key_iv: "56061c1a892441005a6a654373c1887c"
server_key_version: "encryptionkey@0"
I20260619 02:29:32.424530 19882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "5657d994279149cfac5b1b909f4f5b63"
format_stamp: "Formatted at 2026-06-19 02:29:32 on dist-test-slave-4qw6"
server_key: "d25b72e9beb0d4ac4be919fb89724de9"
server_key_iv: "56061c1a892441005a6a654373c1887c"
server_key_version: "encryptionkey@0"
I20260619 02:29:32.429277 19882 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260619 02:29:32.432431 19897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:32.434228 19882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:29:32.434437 19882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "5657d994279149cfac5b1b909f4f5b63"
format_stamp: "Formatted at 2026-06-19 02:29:32 on dist-test-slave-4qw6"
server_key: "d25b72e9beb0d4ac4be919fb89724de9"
server_key_iv: "56061c1a892441005a6a654373c1887c"
server_key_version: "encryptionkey@0"
I20260619 02:29:32.434610 19882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:32.447013 19882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:32.451141 19882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:32.451426 19882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:32.453250 19882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:32.453358 19882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:32.453452 19882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:32.453504 19882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:29:32 dist-test-slave-4qw6 krb5kdc[19485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836172, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:32.468060 19882 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:38173
I20260619 02:29:32.468508 20012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:38173 every 8 connection(s)
I20260619 02:29:32.469419 19882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:29:32.471237 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 19882
I20260619 02:29:32.471391 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:29:32.471835 17795 external_mini_cluster.cc:1428] Setting key f87158c3949afe8661c333d1a35867c3
I20260619 02:29:32.472059 19579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:32.454352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:56869 (local address 127.17.96.254:42351)
0619 02:29:32.454501 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:32.454505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:32.456217 (+ 1712us) server_negotiation.cc:407] Connection header received
0619 02:29:32.457829 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:32.457834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:32.457923 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:32.458026 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:32.460369 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.461404 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.463215 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.463477 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.463615 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:32.467303 (+ 3688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:32.467306 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:32.467309 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:32.467339 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:32.469249 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:32.470013 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:32.470018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:32.470020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:32.470119 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:32.470633 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:32.470637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:32.470638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:32.470870 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:32.470960 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:32.471702 (+ 742us) server_negotiation.cc:299] Negotiation successful
0619 02:29:32.471849 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 02:29:32.473385 19906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:32.455016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:42351 (local address 127.17.96.195:56869)
0619 02:29:32.455947 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:32.456039 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:32.457447 (+ 1408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:32.458430 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:32.458448 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:32.459115 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:32.460081 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:32.460114 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.461595 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.461599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:32.463043 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:32.463057 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.463697 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.463702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:32.463963 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:32.464743 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:32.464763 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:32.467061 (+ 2298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:32.469406 (+ 2345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:32.469413 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:32.469430 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:32.469893 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:32.470251 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:32.470255 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:32.470257 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:32.470493 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:32.471004 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:32.471014 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:32.471393 (+ 379us) client_negotiation.cc:769] Sending connection context
0619 02:29:32.471641 (+ 248us) client_negotiation.cc:240] Negotiation successful
0619 02:29:32.472066 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":203,"threads_started":1}
I20260619 02:29:32.474788 20013 heartbeater.cc:347] Connected to a master server at 127.17.96.254:42351
I20260619 02:29:32.474990 20013 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:32.475603 20013 heartbeater.cc:511] Master 127.17.96.254:42351 requested a full tablet report, sending...
I20260619 02:29:32.476863 19530 ts_manager.cc:194] Registered new tserver with Master: 5657d994279149cfac5b1b909f4f5b63 (127.17.96.195:38173)
I20260619 02:29:32.477823 19530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:56869
I20260619 02:29:32.486433 19579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:32.477379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:51313 (local address 127.17.96.254:42351)
0619 02:29:32.477529 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:32.477534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:32.477738 (+ 204us) server_negotiation.cc:407] Connection header received
0619 02:29:32.477951 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:32.477954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:32.478001 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:32.478099 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:32.478763 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.479293 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.480557 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.480797 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.480901 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:32.482852 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:32.482856 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:32.482861 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:32.482886 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:32.484271 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:32.484849 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:32.484852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:32.484854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:32.484951 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:32.485355 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:32.485359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:32.485361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:32.485582 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:32.485716 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:32.486089 (+ 373us) server_negotiation.cc:299] Negotiation successful
0619 02:29:32.486221 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:29:32.486452 20019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:32.477297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:42351 (local address 127.17.96.195:51313)
0619 02:29:32.477641 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:32.477656 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:32.477826 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:32.478157 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:32.478163 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:32.478426 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:32.478617 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:32.478627 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.479457 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.479461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:32.480394 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:32.480406 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.481139 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.481146 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:32.481345 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:32.482178 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:32.482202 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:32.482616 (+ 414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:32.484397 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:32.484400 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:32.484402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:32.484635 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:32.485081 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:32.485084 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:32.485086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:32.485210 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:32.485717 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:32.485721 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:32.485881 (+ 160us) client_negotiation.cc:769] Sending connection context
0619 02:29:32.486024 (+ 143us) client_negotiation.cc:240] Negotiation successful
0619 02:29:32.486219 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":144,"threads_started":1}
I20260619 02:29:32.486883 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 02:29:32 dist-test-slave-4qw6 krb5kdc[19485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:29:32 dist-test-slave-4qw6 krb5kdc[19485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:32.519865 19579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:32.508925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50616 (local address 127.17.96.254:42351)
0619 02:29:32.509071 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:32.509076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:32.509114 (+ 38us) server_negotiation.cc:407] Connection header received
0619 02:29:32.509184 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:32.509187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:32.509237 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:32.509335 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:32.510293 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.510913 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.512228 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.512744 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.512885 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:32.516083 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:32.516087 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:32.516089 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:32.516127 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:32.517859 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:32.518406 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:32.518411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:32.518413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:32.518514 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:32.518838 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:32.518844 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:32.518846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:32.519076 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:32.519233 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:32.519523 (+ 290us) server_negotiation.cc:299] Negotiation successful
0619 02:29:32.519665 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 02:29:32.523687 19530 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50616:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:29:32.527415 19530 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:29:32.544108 20030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:32.537692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:38173 (local address 127.0.0.1:59350)
0619 02:29:32.538292 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:32.538352 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:32.538504 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:32.539418 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:32.539425 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:32.539455 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:32.539686 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:32.539697 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.541237 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.541241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:32.542477 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:32.542489 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.543460 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.543464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:32.543587 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:32.543634 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 02:29:32.543766 (+ 132us) client_negotiation.cc:240] Negotiation successful
0619 02:29:32.543910 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":4,"thread_start_us":132,"threads_started":1}
I20260619 02:29:32.544152 20033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:32.538047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59350 (local address 127.17.96.195:38173)
0619 02:29:32.538936 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:32.538949 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:32.538971 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:29:32.539038 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:32.539042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:32.539194 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:32.539363 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:32.539868 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.541083 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.542642 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.543315 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.543452 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:32.543654 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:32.543833 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 02:29:32.543956 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":779,"thread_start_us":135,"threads_started":1}
I20260619 02:29:32.545143 20034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:32.538156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:41639 (local address 127.0.0.1:41856)
0619 02:29:32.538619 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:32.538637 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:32.538754 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:32.539709 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:32.539713 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:32.539728 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:32.539875 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:32.539884 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.541471 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.541474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:32.543122 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:32.543128 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.544733 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.544736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:32.544853 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:32.544874 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 02:29:32.544922 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 02:29:32.544995 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":209,"thread_start_us":83,"threads_started":1}
I20260619 02:29:32.545143 20031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:32.537500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53142 (local address 127.17.96.193:45915)
0619 02:29:32.537902 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:32.537910 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:32.538706 (+ 796us) server_negotiation.cc:407] Connection header received
0619 02:29:32.538944 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:32.538952 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:32.539195 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:32.539363 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:32.539954 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.541297 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.543293 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.544258 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.544420 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:32.544585 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:32.544877 (+ 292us) server_negotiation.cc:299] Negotiation successful
0619 02:29:32.544995 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":86,"threads_started":1}
I20260619 02:29:32.545614 20032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:32.537262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:45915 (local address 127.0.0.1:53142)
0619 02:29:32.538608 (+ 1346us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:32.538624 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:32.538782 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:32.539558 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:32.539562 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:32.539582 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:32.539780 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:32.539788 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.541471 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.541474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:32.543114 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:32.543127 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.544577 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.544581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:32.544710 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:32.544737 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 02:29:32.545362 (+ 625us) client_negotiation.cc:240] Negotiation successful
0619 02:29:32.545469 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"mutex_wait_us":174,"thread_start_us":127,"threads_started":1}
I20260619 02:29:32.546293 20035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:32.538613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41856 (local address 127.17.96.194:41639)
0619 02:29:32.539029 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:32.539040 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:32.539061 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:29:32.539130 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:32.539134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:32.539385 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:32.539747 (+ 362us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:32.540013 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.541332 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.543276 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.544599 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.545724 (+ 1125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:32.545944 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:32.546005 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 02:29:32.546109 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":152,"threads_started":1}
I20260619 02:29:32.548126 19947 tablet_service.cc:1467] Processing CreateTablet for tablet 2a8eae0b456143f389171c0cdc2889af (DEFAULT_TABLE table=test-table [id=5f8ff7bfb0ef4be891f73e21aa2bfe57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:32.548792 19667 tablet_service.cc:1467] Processing CreateTablet for tablet 2a8eae0b456143f389171c0cdc2889af (DEFAULT_TABLE table=test-table [id=5f8ff7bfb0ef4be891f73e21aa2bfe57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:32.549561 19947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a8eae0b456143f389171c0cdc2889af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:32.549988 19667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a8eae0b456143f389171c0cdc2889af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:32.552294 19807 tablet_service.cc:1467] Processing CreateTablet for tablet 2a8eae0b456143f389171c0cdc2889af (DEFAULT_TABLE table=test-table [id=5f8ff7bfb0ef4be891f73e21aa2bfe57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:32.553292 19807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a8eae0b456143f389171c0cdc2889af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:32.558897 20036 tablet_bootstrap.cc:492] T 2a8eae0b456143f389171c0cdc2889af P cffd4b4f2c73451196a44fcc9faf826f: Bootstrap starting.
I20260619 02:29:32.559630 20037 tablet_bootstrap.cc:492] T 2a8eae0b456143f389171c0cdc2889af P 5657d994279149cfac5b1b909f4f5b63: Bootstrap starting.
I20260619 02:29:32.562831 20036 tablet_bootstrap.cc:654] T 2a8eae0b456143f389171c0cdc2889af P cffd4b4f2c73451196a44fcc9faf826f: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:32.563045 20037 tablet_bootstrap.cc:654] T 2a8eae0b456143f389171c0cdc2889af P 5657d994279149cfac5b1b909f4f5b63: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:32.563946 20036 log.cc:824] T 2a8eae0b456143f389171c0cdc2889af P cffd4b4f2c73451196a44fcc9faf826f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:32.564337 20037 log.cc:824] T 2a8eae0b456143f389171c0cdc2889af P 5657d994279149cfac5b1b909f4f5b63: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:32.565141 20039 tablet_bootstrap.cc:492] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500: Bootstrap starting.
I20260619 02:29:32.566910 20036 tablet_bootstrap.cc:492] T 2a8eae0b456143f389171c0cdc2889af P cffd4b4f2c73451196a44fcc9faf826f: No bootstrap required, opened a new log
I20260619 02:29:32.566953 20037 tablet_bootstrap.cc:492] T 2a8eae0b456143f389171c0cdc2889af P 5657d994279149cfac5b1b909f4f5b63: No bootstrap required, opened a new log
I20260619 02:29:32.567184 20036 ts_tablet_manager.cc:1397] T 2a8eae0b456143f389171c0cdc2889af P cffd4b4f2c73451196a44fcc9faf826f: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260619 02:29:32.567185 20037 ts_tablet_manager.cc:1397] T 2a8eae0b456143f389171c0cdc2889af P 5657d994279149cfac5b1b909f4f5b63: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260619 02:29:32.568737 20039 tablet_bootstrap.cc:654] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:32.569855 20039 log.cc:824] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:32.571935 20037 raft_consensus.cc:348] T 2a8eae0b456143f389171c0cdc2889af P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:32.571977 20036 raft_consensus.cc:348] T 2a8eae0b456143f389171c0cdc2889af P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:32.572353 20037 raft_consensus.cc:374] T 2a8eae0b456143f389171c0cdc2889af P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:32.572373 20036 raft_consensus.cc:374] T 2a8eae0b456143f389171c0cdc2889af P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:32.572441 20036 raft_consensus.cc:729] T 2a8eae0b456143f389171c0cdc2889af P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cffd4b4f2c73451196a44fcc9faf826f, State: Initialized, Role: FOLLOWER
I20260619 02:29:32.572429 20037 raft_consensus.cc:729] T 2a8eae0b456143f389171c0cdc2889af P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5657d994279149cfac5b1b909f4f5b63, State: Initialized, Role: FOLLOWER
I20260619 02:29:32.572606 20039 tablet_bootstrap.cc:492] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500: No bootstrap required, opened a new log
I20260619 02:29:32.572881 20039 ts_tablet_manager.cc:1397] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260619 02:29:32.573007 20036 consensus_queue.cc:260] T 2a8eae0b456143f389171c0cdc2889af P cffd4b4f2c73451196a44fcc9faf826f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:32.572993 20037 consensus_queue.cc:260] T 2a8eae0b456143f389171c0cdc2889af P 5657d994279149cfac5b1b909f4f5b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:32.573890 20013 heartbeater.cc:503] Master 127.17.96.254:42351 was elected leader, sending a full tablet report...
I20260619 02:29:32.574199 20036 ts_tablet_manager.cc:1428] T 2a8eae0b456143f389171c0cdc2889af P cffd4b4f2c73451196a44fcc9faf826f: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260619 02:29:32.574468 20037 ts_tablet_manager.cc:1428] T 2a8eae0b456143f389171c0cdc2889af P 5657d994279149cfac5b1b909f4f5b63: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260619 02:29:32.574616 19733 heartbeater.cc:503] Master 127.17.96.254:42351 was elected leader, sending a full tablet report...
I20260619 02:29:32.578454 20039 raft_consensus.cc:348] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:32.578881 20039 raft_consensus.cc:374] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:32.578949 20039 raft_consensus.cc:729] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8223df6185ea409ebbdd26b192a64500, State: Initialized, Role: FOLLOWER
I20260619 02:29:32.579511 20039 consensus_queue.cc:260] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:32.580753 20039 ts_tablet_manager.cc:1428] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260619 02:29:32.581136 19873 heartbeater.cc:503] Master 127.17.96.254:42351 was elected leader, sending a full tablet report...
W20260619 02:29:32.684238 19734 tablet.cc:2369] T 2a8eae0b456143f389171c0cdc2889af P cffd4b4f2c73451196a44fcc9faf826f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:29:32.704097 19874 tablet.cc:2369] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:29:32.721171 20014 tablet.cc:2369] T 2a8eae0b456143f389171c0cdc2889af P 5657d994279149cfac5b1b909f4f5b63: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:29:32.889824 20044 raft_consensus.cc:482] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:29:32.890066 20044 raft_consensus.cc:504] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:32.891440 20044 leader_election.cc:288] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cffd4b4f2c73451196a44fcc9faf826f (127.17.96.193:45915), 5657d994279149cfac5b1b909f4f5b63 (127.17.96.195:38173)
I20260619 02:29:32.896103 20033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:32.892098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:44139 (local address 127.17.96.195:38173)
0619 02:29:32.892216 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:32.892221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:32.892370 (+ 149us) server_negotiation.cc:407] Connection header received
0619 02:29:32.892506 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:32.892510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:32.892568 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:32.892674 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:32.892992 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.893597 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.894939 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.895403 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.895518 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:32.895612 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:32.895842 (+ 230us) server_negotiation.cc:299] Negotiation successful
0619 02:29:32.895944 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 02:29:32.896132 20045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:32.891808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:45915 (local address 127.17.96.194:41167)
0619 02:29:32.892356 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:32.892385 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:32.892508 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:32.892840 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:32.892842 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:32.892857 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:32.892978 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:32.892985 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.893843 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.893846 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:32.894801 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:32.894811 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.895640 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.895645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:32.895748 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:32.895767 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 02:29:32.895879 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 02:29:32.895954 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":102,"thread_start_us":104,"threads_started":1}
I20260619 02:29:32.896145 20031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:32.891981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:41167 (local address 127.17.96.193:45915)
0619 02:29:32.892137 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:32.892142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:32.892496 (+ 354us) server_negotiation.cc:407] Connection header received
0619 02:29:32.892641 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:32.892648 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:32.892706 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:32.892795 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:32.893103 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.893631 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.894989 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.895481 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.895618 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:32.895678 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:32.895918 (+ 240us) server_negotiation.cc:299] Negotiation successful
0619 02:29:32.896024 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 02:29:32.896139 20046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:32.891808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:38173 (local address 127.17.96.194:44139)
0619 02:29:32.892260 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:32.892274 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:32.892387 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:32.892679 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:32.892682 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:32.892700 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:32.892853 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:32.892861 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.893730 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.893737 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:32.894783 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:32.894795 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:32.895556 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:32.895564 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:32.895675 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:32.895714 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 02:29:32.895795 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 02:29:32.895902 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":39,"thread_start_us":115,"threads_started":1}
I20260619 02:29:32.896862 19967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a8eae0b456143f389171c0cdc2889af" candidate_uuid: "8223df6185ea409ebbdd26b192a64500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5657d994279149cfac5b1b909f4f5b63" is_pre_election: true
I20260619 02:29:32.896860 19687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a8eae0b456143f389171c0cdc2889af" candidate_uuid: "8223df6185ea409ebbdd26b192a64500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cffd4b4f2c73451196a44fcc9faf826f" is_pre_election: true
I20260619 02:29:32.897202 19687 raft_consensus.cc:2454] T 2a8eae0b456143f389171c0cdc2889af P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8223df6185ea409ebbdd26b192a64500 in term 0.
I20260619 02:29:32.897217 19967 raft_consensus.cc:2454] T 2a8eae0b456143f389171c0cdc2889af P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8223df6185ea409ebbdd26b192a64500 in term 0.
I20260619 02:29:32.897792 19761 leader_election.cc:302] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5657d994279149cfac5b1b909f4f5b63, 8223df6185ea409ebbdd26b192a64500; no voters:
I20260619 02:29:32.898108 20044 raft_consensus.cc:2781] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:29:32.898195 20044 raft_consensus.cc:482] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:29:32.898231 20044 raft_consensus.cc:3037] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:32.899464 20044 raft_consensus.cc:504] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:32.899971 20044 leader_election.cc:288] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500 [CANDIDATE]: Term 1 election: Requested vote from peers cffd4b4f2c73451196a44fcc9faf826f (127.17.96.193:45915), 5657d994279149cfac5b1b909f4f5b63 (127.17.96.195:38173)
I20260619 02:29:32.900575 19687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a8eae0b456143f389171c0cdc2889af" candidate_uuid: "8223df6185ea409ebbdd26b192a64500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cffd4b4f2c73451196a44fcc9faf826f"
I20260619 02:29:32.900734 19967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a8eae0b456143f389171c0cdc2889af" candidate_uuid: "8223df6185ea409ebbdd26b192a64500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5657d994279149cfac5b1b909f4f5b63"
I20260619 02:29:32.900879 19967 raft_consensus.cc:3037] T 2a8eae0b456143f389171c0cdc2889af P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:32.900832 19687 raft_consensus.cc:3037] T 2a8eae0b456143f389171c0cdc2889af P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:32.902570 19967 raft_consensus.cc:2454] T 2a8eae0b456143f389171c0cdc2889af P 5657d994279149cfac5b1b909f4f5b63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8223df6185ea409ebbdd26b192a64500 in term 1.
I20260619 02:29:32.902765 19687 raft_consensus.cc:2454] T 2a8eae0b456143f389171c0cdc2889af P cffd4b4f2c73451196a44fcc9faf826f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8223df6185ea409ebbdd26b192a64500 in term 1.
I20260619 02:29:32.903003 19761 leader_election.cc:302] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5657d994279149cfac5b1b909f4f5b63, 8223df6185ea409ebbdd26b192a64500; no voters:
I20260619 02:29:32.903227 20044 raft_consensus.cc:2781] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:32.903491 20044 raft_consensus.cc:686] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500 [term 1 LEADER]: Becoming Leader. State: Replica: 8223df6185ea409ebbdd26b192a64500, State: Running, Role: LEADER
I20260619 02:29:32.903877 20044 consensus_queue.cc:237] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:32.907469 19530 catalog_manager.cc:5495] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8223df6185ea409ebbdd26b192a64500 (127.17.96.194). New cstate: current_term: 1 leader_uuid: "8223df6185ea409ebbdd26b192a64500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } health_report { overall_health: UNKNOWN } } }
Jun 19 02:29:33 dist-test-slave-4qw6 krb5kdc[19485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836171, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:33.034538 20054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:33.023833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:42351 (local address 127.0.0.1:50632)
0619 02:29:33.024242 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:33.024258 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:33.024385 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:33.024676 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:33.024678 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:33.025133 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:33.025273 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:33.025281 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:33.026174 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:33.026176 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:33.026763 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:33.026772 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:33.027529 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:33.027534 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:33.027656 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:33.028522 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:33.028536 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:33.030435 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:33.032536 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:33.032549 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:33.032556 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:33.032849 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:33.033294 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:33.033296 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:33.033298 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:33.033387 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:33.033920 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:33.033926 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:33.034076 (+ 150us) client_negotiation.cc:769] Sending connection context
0619 02:29:33.034191 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 02:29:33.034347 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":119,"threads_started":1}
I20260619 02:29:33.034727 20055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:33.023934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50632 (local address 127.17.96.254:42351)
0619 02:29:33.024250 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:33.024257 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:33.024353 (+ 96us) server_negotiation.cc:407] Connection header received
0619 02:29:33.024482 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:33.024485 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:33.024532 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:33.024617 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:33.025397 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:33.026069 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:33.027052 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:33.027319 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:33.027414 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:33.030678 (+ 3264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:33.030681 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:33.030684 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:33.030711 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:33.032376 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:33.033022 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:33.033027 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:33.033028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:33.033175 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:33.033481 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:33.033484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:33.033485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:33.033727 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:33.033912 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:33.034268 (+ 356us) server_negotiation.cc:299] Negotiation successful
0619 02:29:33.034429 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":166,"threads_started":1}
I20260619 02:29:33.040134 19530 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:50632:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 02:29:33.041115 19530 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:29:33.047309 19667 tablet_service.cc:1467] Processing CreateTablet for tablet 320f15dfbed441ad8494213402186a75 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b03f92927baf4a99896d05ce8330efa5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:29:33.047335 19947 tablet_service.cc:1467] Processing CreateTablet for tablet 320f15dfbed441ad8494213402186a75 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b03f92927baf4a99896d05ce8330efa5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:29:33.047169 19807 tablet_service.cc:1467] Processing CreateTablet for tablet 320f15dfbed441ad8494213402186a75 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b03f92927baf4a99896d05ce8330efa5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:29:33.048030 19667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 320f15dfbed441ad8494213402186a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:33.048024 19807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 320f15dfbed441ad8494213402186a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:33.048025 19947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 320f15dfbed441ad8494213402186a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:33.051302 20036 tablet_bootstrap.cc:492] T 320f15dfbed441ad8494213402186a75 P cffd4b4f2c73451196a44fcc9faf826f: Bootstrap starting.
I20260619 02:29:33.051453 20039 tablet_bootstrap.cc:492] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500: Bootstrap starting.
I20260619 02:29:33.052322 20036 tablet_bootstrap.cc:654] T 320f15dfbed441ad8494213402186a75 P cffd4b4f2c73451196a44fcc9faf826f: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:33.052474 20039 tablet_bootstrap.cc:654] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:33.053709 20037 tablet_bootstrap.cc:492] T 320f15dfbed441ad8494213402186a75 P 5657d994279149cfac5b1b909f4f5b63: Bootstrap starting.
I20260619 02:29:33.054680 20039 tablet_bootstrap.cc:492] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500: No bootstrap required, opened a new log
I20260619 02:29:33.054843 20039 ts_tablet_manager.cc:1397] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 02:29:33.054844 20037 tablet_bootstrap.cc:654] T 320f15dfbed441ad8494213402186a75 P 5657d994279149cfac5b1b909f4f5b63: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:33.055502 20039 raft_consensus.cc:348] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:33.055662 20039 raft_consensus.cc:374] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:33.055711 20039 raft_consensus.cc:729] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8223df6185ea409ebbdd26b192a64500, State: Initialized, Role: FOLLOWER
I20260619 02:29:33.055909 20039 consensus_queue.cc:260] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:33.056252 20044 tablet_replica.cc:439] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } } }
I20260619 02:29:33.056434 20044 tablet_replica.cc:442] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:29:33.057144 20037 tablet_bootstrap.cc:492] T 320f15dfbed441ad8494213402186a75 P 5657d994279149cfac5b1b909f4f5b63: No bootstrap required, opened a new log
I20260619 02:29:33.057169 20039 ts_tablet_manager.cc:1428] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:29:33.057262 20037 ts_tablet_manager.cc:1397] T 320f15dfbed441ad8494213402186a75 P 5657d994279149cfac5b1b909f4f5b63: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 02:29:33.057698 20037 raft_consensus.cc:348] T 320f15dfbed441ad8494213402186a75 P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:33.057878 20037 raft_consensus.cc:374] T 320f15dfbed441ad8494213402186a75 P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:33.057945 20037 raft_consensus.cc:729] T 320f15dfbed441ad8494213402186a75 P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5657d994279149cfac5b1b909f4f5b63, State: Initialized, Role: FOLLOWER
I20260619 02:29:33.058094 20037 consensus_queue.cc:260] T 320f15dfbed441ad8494213402186a75 P 5657d994279149cfac5b1b909f4f5b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:33.058341 20036 tablet_bootstrap.cc:492] T 320f15dfbed441ad8494213402186a75 P cffd4b4f2c73451196a44fcc9faf826f: No bootstrap required, opened a new log
I20260619 02:29:33.058400 20037 ts_tablet_manager.cc:1428] T 320f15dfbed441ad8494213402186a75 P 5657d994279149cfac5b1b909f4f5b63: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:29:33.058454 20036 ts_tablet_manager.cc:1397] T 320f15dfbed441ad8494213402186a75 P cffd4b4f2c73451196a44fcc9faf826f: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260619 02:29:33.058461 20043 tablet_replica.cc:439] T 320f15dfbed441ad8494213402186a75 P 5657d994279149cfac5b1b909f4f5b63: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } } }
I20260619 02:29:33.058647 20043 tablet_replica.cc:442] T 320f15dfbed441ad8494213402186a75 P 5657d994279149cfac5b1b909f4f5b63: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:29:33.059047 20036 raft_consensus.cc:348] T 320f15dfbed441ad8494213402186a75 P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:33.059638 20036 raft_consensus.cc:374] T 320f15dfbed441ad8494213402186a75 P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:33.059688 20036 raft_consensus.cc:729] T 320f15dfbed441ad8494213402186a75 P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cffd4b4f2c73451196a44fcc9faf826f, State: Initialized, Role: FOLLOWER
I20260619 02:29:33.059867 20036 consensus_queue.cc:260] T 320f15dfbed441ad8494213402186a75 P cffd4b4f2c73451196a44fcc9faf826f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:33.060242 20042 tablet_replica.cc:439] T 320f15dfbed441ad8494213402186a75 P cffd4b4f2c73451196a44fcc9faf826f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } } }
I20260619 02:29:33.060429 20042 tablet_replica.cc:442] T 320f15dfbed441ad8494213402186a75 P cffd4b4f2c73451196a44fcc9faf826f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:29:33.060607 20036 ts_tablet_manager.cc:1428] T 320f15dfbed441ad8494213402186a75 P cffd4b4f2c73451196a44fcc9faf826f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:29:33.115442 20044 raft_consensus.cc:482] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:29:33.115610 20044 raft_consensus.cc:504] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:33.116185 20044 leader_election.cc:288] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cffd4b4f2c73451196a44fcc9faf826f (127.17.96.193:45915), 5657d994279149cfac5b1b909f4f5b63 (127.17.96.195:38173)
I20260619 02:29:33.116793 19967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "320f15dfbed441ad8494213402186a75" candidate_uuid: "8223df6185ea409ebbdd26b192a64500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5657d994279149cfac5b1b909f4f5b63" is_pre_election: true
I20260619 02:29:33.116806 19687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "320f15dfbed441ad8494213402186a75" candidate_uuid: "8223df6185ea409ebbdd26b192a64500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cffd4b4f2c73451196a44fcc9faf826f" is_pre_election: true
I20260619 02:29:33.116978 19967 raft_consensus.cc:2454] T 320f15dfbed441ad8494213402186a75 P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8223df6185ea409ebbdd26b192a64500 in term 0.
I20260619 02:29:33.116987 19687 raft_consensus.cc:2454] T 320f15dfbed441ad8494213402186a75 P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8223df6185ea409ebbdd26b192a64500 in term 0.
I20260619 02:29:33.117513 19761 leader_election.cc:302] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5657d994279149cfac5b1b909f4f5b63, 8223df6185ea409ebbdd26b192a64500, cffd4b4f2c73451196a44fcc9faf826f; no voters:
I20260619 02:29:33.117736 20044 raft_consensus.cc:2781] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:29:33.117832 20044 raft_consensus.cc:482] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:29:33.117866 20044 raft_consensus.cc:3037] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:33.118932 20044 raft_consensus.cc:504] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:33.119366 20044 leader_election.cc:288] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500 [CANDIDATE]: Term 1 election: Requested vote from peers cffd4b4f2c73451196a44fcc9faf826f (127.17.96.193:45915), 5657d994279149cfac5b1b909f4f5b63 (127.17.96.195:38173)
I20260619 02:29:33.119840 19687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "320f15dfbed441ad8494213402186a75" candidate_uuid: "8223df6185ea409ebbdd26b192a64500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cffd4b4f2c73451196a44fcc9faf826f"
I20260619 02:29:33.119840 19967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "320f15dfbed441ad8494213402186a75" candidate_uuid: "8223df6185ea409ebbdd26b192a64500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5657d994279149cfac5b1b909f4f5b63"
I20260619 02:29:33.119964 19687 raft_consensus.cc:3037] T 320f15dfbed441ad8494213402186a75 P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:33.119998 19967 raft_consensus.cc:3037] T 320f15dfbed441ad8494213402186a75 P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:33.121014 19967 raft_consensus.cc:2454] T 320f15dfbed441ad8494213402186a75 P 5657d994279149cfac5b1b909f4f5b63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8223df6185ea409ebbdd26b192a64500 in term 1.
I20260619 02:29:33.121410 19761 leader_election.cc:302] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5657d994279149cfac5b1b909f4f5b63, 8223df6185ea409ebbdd26b192a64500; no voters:
I20260619 02:29:33.121596 19687 raft_consensus.cc:2454] T 320f15dfbed441ad8494213402186a75 P cffd4b4f2c73451196a44fcc9faf826f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8223df6185ea409ebbdd26b192a64500 in term 1.
I20260619 02:29:33.121682 20044 raft_consensus.cc:2781] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:33.121829 20044 raft_consensus.cc:686] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500 [term 1 LEADER]: Becoming Leader. State: Replica: 8223df6185ea409ebbdd26b192a64500, State: Running, Role: LEADER
I20260619 02:29:33.122018 20044 consensus_queue.cc:237] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:33.122797 20047 tablet_replica.cc:439] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500: TxnStatusTablet state changed. Reason: New leader 8223df6185ea409ebbdd26b192a64500. Latest consensus state: current_term: 1 leader_uuid: "8223df6185ea409ebbdd26b192a64500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } } }
I20260619 02:29:33.122913 20047 tablet_replica.cc:442] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:29:33.123261 20060 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 02:29:33.123534 20060 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 02:29:33.124107 19525 catalog_manager.cc:5495] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8223df6185ea409ebbdd26b192a64500 (127.17.96.194). New cstate: current_term: 1 leader_uuid: "8223df6185ea409ebbdd26b192a64500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } health_report { overall_health: UNKNOWN } } }
I20260619 02:29:33.147125 20054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:33.142028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:41639 (local address 127.0.0.1:41868)
0619 02:29:33.142238 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:33.142261 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:33.142407 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:33.142925 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:33.142928 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:33.142935 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:29:33.143046 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:33.143050 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:33.144248 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:33.144251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:33.144935 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:33.144944 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:33.145620 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:33.145622 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:33.145702 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:33.145783 (+ 81us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:29:33.146826 (+ 1043us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:29:33.146840 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 02:29:33.146899 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 02:29:33.146985 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260619 02:29:33.147207 20063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:33.142140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41868 (local address 127.17.96.194:41639)
0619 02:29:33.142525 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:33.142536 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:33.142558 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:29:33.142654 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:33.142657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:33.142763 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:33.142908 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:29:33.143164 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:33.144095 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:33.145070 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:33.145483 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:33.145612 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:33.146028 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:29:33.146712 (+ 684us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:29:33.146848 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:33.146918 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 02:29:33.146987 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":155,"threads_started":1}
I20260619 02:29:33.154889 19967 raft_consensus.cc:1261] T 320f15dfbed441ad8494213402186a75 P 5657d994279149cfac5b1b909f4f5b63 [term 1 FOLLOWER]: Refusing update from remote peer 8223df6185ea409ebbdd26b192a64500: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:29:33.154889 19687 raft_consensus.cc:1261] T 320f15dfbed441ad8494213402186a75 P cffd4b4f2c73451196a44fcc9faf826f [term 1 FOLLOWER]: Refusing update from remote peer 8223df6185ea409ebbdd26b192a64500: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:29:33.155891 20044 consensus_queue.cc:1035] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:29:33.156208 20047 consensus_queue.cc:1035] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:29:33.161448 20042 tablet_replica.cc:439] T 320f15dfbed441ad8494213402186a75 P cffd4b4f2c73451196a44fcc9faf826f: TxnStatusTablet state changed. Reason: New leader 8223df6185ea409ebbdd26b192a64500. Latest consensus state: current_term: 1 leader_uuid: "8223df6185ea409ebbdd26b192a64500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } } }
I20260619 02:29:33.161676 20042 tablet_replica.cc:442] T 320f15dfbed441ad8494213402186a75 P cffd4b4f2c73451196a44fcc9faf826f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:29:33.163900 20043 tablet_replica.cc:439] T 320f15dfbed441ad8494213402186a75 P 5657d994279149cfac5b1b909f4f5b63: TxnStatusTablet state changed. Reason: New leader 8223df6185ea409ebbdd26b192a64500. Latest consensus state: current_term: 1 leader_uuid: "8223df6185ea409ebbdd26b192a64500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } } }
I20260619 02:29:33.164119 20043 tablet_replica.cc:442] T 320f15dfbed441ad8494213402186a75 P 5657d994279149cfac5b1b909f4f5b63: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:29:33.166880 20047 tablet_replica.cc:439] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8223df6185ea409ebbdd26b192a64500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } } }
I20260619 02:29:33.167074 20064 mvcc.cc:205] Tried to move back new op lower bound from 7298400965231869952 to 7298400965109518336. Current Snapshot: MvccSnapshot[applied={T|T < 7298400965231869952}]
I20260619 02:29:33.167088 20047 tablet_replica.cc:442] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:29:33.168787 20065 mvcc.cc:205] Tried to move back new op lower bound from 7298400965231869952 to 7298400965109518336. Current Snapshot: MvccSnapshot[applied={T|T < 7298400965231869952}]
I20260619 02:29:33.169039 20042 tablet_replica.cc:439] T 320f15dfbed441ad8494213402186a75 P cffd4b4f2c73451196a44fcc9faf826f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8223df6185ea409ebbdd26b192a64500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } } }
I20260619 02:29:33.169188 20042 tablet_replica.cc:442] T 320f15dfbed441ad8494213402186a75 P cffd4b4f2c73451196a44fcc9faf826f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:29:33.172598 20047 tablet_replica.cc:439] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8223df6185ea409ebbdd26b192a64500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } } }
I20260619 02:29:33.172762 20047 tablet_replica.cc:442] T 320f15dfbed441ad8494213402186a75 P 8223df6185ea409ebbdd26b192a64500: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:29:33.175395 20043 tablet_replica.cc:439] T 320f15dfbed441ad8494213402186a75 P 5657d994279149cfac5b1b909f4f5b63: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8223df6185ea409ebbdd26b192a64500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } } }
I20260619 02:29:33.175596 20043 tablet_replica.cc:442] T 320f15dfbed441ad8494213402186a75 P 5657d994279149cfac5b1b909f4f5b63: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:29:33.181252 20063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:33.176078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41870 (local address 127.17.96.194:41639)
0619 02:29:33.176236 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:33.176242 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:33.176825 (+ 583us) server_negotiation.cc:407] Connection header received
0619 02:29:33.176994 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:33.176998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:33.177055 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:33.177141 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:29:33.177820 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:33.178696 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:33.179778 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:33.180161 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:33.180303 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:33.180645 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:29:33.180753 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:29:33.180850 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:33.181005 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 02:29:33.181072 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 02:29:33.194733 20063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:33.190767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:42775 (local address 127.17.96.194:41639)
0619 02:29:33.190920 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:33.190925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:33.191137 (+ 212us) server_negotiation.cc:407] Connection header received
0619 02:29:33.191301 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:33.191304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:33.191352 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:33.191440 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:29:33.191709 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:33.192252 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:33.193141 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:33.193357 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:33.193485 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:33.194131 (+ 646us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:29:33.194229 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:29:33.194329 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:33.194432 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 02:29:33.194508 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":47}
I20260619 02:29:33.194742 20076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:33.190708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:41639 (local address 127.17.96.194:42775)
0619 02:29:33.191041 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:33.191054 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:33.191198 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:33.191446 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:33.191449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:33.191456 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:29:33.191605 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:33.191610 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:33.192370 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:33.192374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:33.193014 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:33.193023 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:33.193495 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:33.193499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:33.193838 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:33.193913 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:29:33.194330 (+ 417us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:29:33.194338 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 02:29:33.194414 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 02:29:33.194541 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":24,"thread_start_us":123,"threads_started":1}
I20260619 02:29:33.204325 19967 raft_consensus.cc:1261] T 2a8eae0b456143f389171c0cdc2889af P 5657d994279149cfac5b1b909f4f5b63 [term 1 FOLLOWER]: Refusing update from remote peer 8223df6185ea409ebbdd26b192a64500: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:29:33.204701 19687 raft_consensus.cc:1261] T 2a8eae0b456143f389171c0cdc2889af P cffd4b4f2c73451196a44fcc9faf826f [term 1 FOLLOWER]: Refusing update from remote peer 8223df6185ea409ebbdd26b192a64500: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:29:33.205159 20044 consensus_queue.cc:1035] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:29:33.205587 20047 consensus_queue.cc:1035] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:29:33.267524 19525 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:29:33.267824 19525 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:29:33.271929 19525 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714: Sending DeleteTablet for 3 replicas of tablet 2a8eae0b456143f389171c0cdc2889af
I20260619 02:29:33.273101 19807 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a8eae0b456143f389171c0cdc2889af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:33 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:41856
I20260619 02:29:33.273097 19667 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a8eae0b456143f389171c0cdc2889af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:33 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:53142
I20260619 02:29:33.273094 19947 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a8eae0b456143f389171c0cdc2889af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:33 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:59350
I20260619 02:29:33.275297 20082 tablet_replica.cc:330] T 2a8eae0b456143f389171c0cdc2889af P 5657d994279149cfac5b1b909f4f5b63: stopping tablet replica
I20260619 02:29:33.275763 20082 raft_consensus.cc:2229] T 2a8eae0b456143f389171c0cdc2889af P 5657d994279149cfac5b1b909f4f5b63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:29:33.276091 20082 raft_consensus.cc:2258] T 2a8eae0b456143f389171c0cdc2889af P 5657d994279149cfac5b1b909f4f5b63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:33.276609 19527 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50616:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:29:33.277266 19527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:29:33.277251 20083 tablet_replica.cc:330] T 2a8eae0b456143f389171c0cdc2889af P cffd4b4f2c73451196a44fcc9faf826f: stopping tablet replica
I20260619 02:29:33.278023 20083 raft_consensus.cc:2229] T 2a8eae0b456143f389171c0cdc2889af P cffd4b4f2c73451196a44fcc9faf826f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:29:33.278394 20083 raft_consensus.cc:2258] T 2a8eae0b456143f389171c0cdc2889af P cffd4b4f2c73451196a44fcc9faf826f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:33.282603 20083 ts_tablet_manager.cc:1905] T 2a8eae0b456143f389171c0cdc2889af P cffd4b4f2c73451196a44fcc9faf826f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:33.285085 19807 tablet_service.cc:1467] Processing CreateTablet for tablet 8f671dce5d464b7e91d8a81836b668ec (DEFAULT_TABLE table=test-table [id=eb08b9102b4749f49ce798cf88f9916c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:33.285240 19667 tablet_service.cc:1467] Processing CreateTablet for tablet 8f671dce5d464b7e91d8a81836b668ec (DEFAULT_TABLE table=test-table [id=eb08b9102b4749f49ce798cf88f9916c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:33.285553 19807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f671dce5d464b7e91d8a81836b668ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:33.285665 19667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f671dce5d464b7e91d8a81836b668ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:33.289841 19947 tablet_service.cc:1467] Processing CreateTablet for tablet 8f671dce5d464b7e91d8a81836b668ec (DEFAULT_TABLE table=test-table [id=eb08b9102b4749f49ce798cf88f9916c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:33.290268 19947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f671dce5d464b7e91d8a81836b668ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:33.290424 20036 tablet_bootstrap.cc:492] T 8f671dce5d464b7e91d8a81836b668ec P cffd4b4f2c73451196a44fcc9faf826f: Bootstrap starting.
I20260619 02:29:33.291416 20082 ts_tablet_manager.cc:1905] T 2a8eae0b456143f389171c0cdc2889af P 5657d994279149cfac5b1b909f4f5b63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:33.291572 20036 tablet_bootstrap.cc:654] T 8f671dce5d464b7e91d8a81836b668ec P cffd4b4f2c73451196a44fcc9faf826f: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:33.293190 20036 tablet_bootstrap.cc:492] T 8f671dce5d464b7e91d8a81836b668ec P cffd4b4f2c73451196a44fcc9faf826f: No bootstrap required, opened a new log
I20260619 02:29:33.293310 20036 ts_tablet_manager.cc:1397] T 8f671dce5d464b7e91d8a81836b668ec P cffd4b4f2c73451196a44fcc9faf826f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 02:29:33.293946 20036 raft_consensus.cc:348] T 8f671dce5d464b7e91d8a81836b668ec P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:33.294112 20036 raft_consensus.cc:374] T 8f671dce5d464b7e91d8a81836b668ec P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:33.294157 20036 raft_consensus.cc:729] T 8f671dce5d464b7e91d8a81836b668ec P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cffd4b4f2c73451196a44fcc9faf826f, State: Initialized, Role: FOLLOWER
I20260619 02:29:33.294307 20036 consensus_queue.cc:260] T 8f671dce5d464b7e91d8a81836b668ec P cffd4b4f2c73451196a44fcc9faf826f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:33.294660 20036 ts_tablet_manager.cc:1428] T 8f671dce5d464b7e91d8a81836b668ec P cffd4b4f2c73451196a44fcc9faf826f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:29:33.297258 20082 ts_tablet_manager.cc:1918] T 2a8eae0b456143f389171c0cdc2889af P 5657d994279149cfac5b1b909f4f5b63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:29:33.297420 20082 log.cc:1196] T 2a8eae0b456143f389171c0cdc2889af P 5657d994279149cfac5b1b909f4f5b63: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/2a8eae0b456143f389171c0cdc2889af
I20260619 02:29:33.297847 20082 ts_tablet_manager.cc:1939] T 2a8eae0b456143f389171c0cdc2889af P 5657d994279149cfac5b1b909f4f5b63: Deleting consensus metadata
I20260619 02:29:33.299661 20039 tablet_bootstrap.cc:492] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500: Bootstrap starting.
I20260619 02:29:33.299636 19517 catalog_manager.cc:4841] TS 5657d994279149cfac5b1b909f4f5b63 (127.17.96.195:38173): tablet 2a8eae0b456143f389171c0cdc2889af (table test-table [id=5f8ff7bfb0ef4be891f73e21aa2bfe57]) successfully deleted
I20260619 02:29:33.301064 20039 tablet_bootstrap.cc:654] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:33.301594 20083 ts_tablet_manager.cc:1918] T 2a8eae0b456143f389171c0cdc2889af P cffd4b4f2c73451196a44fcc9faf826f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:29:33.302347 20083 log.cc:1196] T 2a8eae0b456143f389171c0cdc2889af P cffd4b4f2c73451196a44fcc9faf826f: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-0/wal/wals/2a8eae0b456143f389171c0cdc2889af
I20260619 02:29:33.302734 20083 ts_tablet_manager.cc:1939] T 2a8eae0b456143f389171c0cdc2889af P cffd4b4f2c73451196a44fcc9faf826f: Deleting consensus metadata
I20260619 02:29:33.302910 20039 tablet_bootstrap.cc:492] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500: No bootstrap required, opened a new log
I20260619 02:29:33.303038 20039 ts_tablet_manager.cc:1397] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260619 02:29:33.303673 20039 raft_consensus.cc:348] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:33.303853 20039 raft_consensus.cc:374] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:33.303905 20039 raft_consensus.cc:729] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8223df6185ea409ebbdd26b192a64500, State: Initialized, Role: FOLLOWER
I20260619 02:29:33.304373 19516 catalog_manager.cc:4841] TS cffd4b4f2c73451196a44fcc9faf826f (127.17.96.193:45915): tablet 2a8eae0b456143f389171c0cdc2889af (table test-table [id=5f8ff7bfb0ef4be891f73e21aa2bfe57]) successfully deleted
I20260619 02:29:33.304940 20039 consensus_queue.cc:260] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:33.305397 20037 tablet_bootstrap.cc:492] T 8f671dce5d464b7e91d8a81836b668ec P 5657d994279149cfac5b1b909f4f5b63: Bootstrap starting.
I20260619 02:29:33.305526 20039 ts_tablet_manager.cc:1428] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:29:33.306847 20037 tablet_bootstrap.cc:654] T 8f671dce5d464b7e91d8a81836b668ec P 5657d994279149cfac5b1b909f4f5b63: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:33.306984 20081 tablet_replica.cc:330] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500: stopping tablet replica
I20260619 02:29:33.307422 20081 raft_consensus.cc:2229] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:29:33.308120 20081 raft_consensus.cc:2258] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:33.309892 20081 ts_tablet_manager.cc:1905] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:33.310855 20037 tablet_bootstrap.cc:492] T 8f671dce5d464b7e91d8a81836b668ec P 5657d994279149cfac5b1b909f4f5b63: No bootstrap required, opened a new log
I20260619 02:29:33.310981 20037 ts_tablet_manager.cc:1397] T 8f671dce5d464b7e91d8a81836b668ec P 5657d994279149cfac5b1b909f4f5b63: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260619 02:29:33.311607 20037 raft_consensus.cc:348] T 8f671dce5d464b7e91d8a81836b668ec P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:33.311807 20037 raft_consensus.cc:374] T 8f671dce5d464b7e91d8a81836b668ec P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:33.311884 20037 raft_consensus.cc:729] T 8f671dce5d464b7e91d8a81836b668ec P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5657d994279149cfac5b1b909f4f5b63, State: Initialized, Role: FOLLOWER
I20260619 02:29:33.312709 20037 consensus_queue.cc:260] T 8f671dce5d464b7e91d8a81836b668ec P 5657d994279149cfac5b1b909f4f5b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:33.313128 20037 ts_tablet_manager.cc:1428] T 8f671dce5d464b7e91d8a81836b668ec P 5657d994279149cfac5b1b909f4f5b63: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 02:29:33.313948 20081 ts_tablet_manager.cc:1918] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:29:33.314074 20081 log.cc:1196] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/2a8eae0b456143f389171c0cdc2889af
I20260619 02:29:33.314507 20081 ts_tablet_manager.cc:1939] T 2a8eae0b456143f389171c0cdc2889af P 8223df6185ea409ebbdd26b192a64500: Deleting consensus metadata
I20260619 02:29:33.316015 19516 catalog_manager.cc:4841] TS 8223df6185ea409ebbdd26b192a64500 (127.17.96.194:41639): tablet 2a8eae0b456143f389171c0cdc2889af (table test-table [id=5f8ff7bfb0ef4be891f73e21aa2bfe57]) successfully deleted
I20260619 02:29:33.441298 20044 raft_consensus.cc:482] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:29:33.441460 20044 raft_consensus.cc:504] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:33.442087 20044 leader_election.cc:288] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cffd4b4f2c73451196a44fcc9faf826f (127.17.96.193:45915), 5657d994279149cfac5b1b909f4f5b63 (127.17.96.195:38173)
I20260619 02:29:33.442632 19967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f671dce5d464b7e91d8a81836b668ec" candidate_uuid: "8223df6185ea409ebbdd26b192a64500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5657d994279149cfac5b1b909f4f5b63" is_pre_election: true
I20260619 02:29:33.442675 19687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f671dce5d464b7e91d8a81836b668ec" candidate_uuid: "8223df6185ea409ebbdd26b192a64500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cffd4b4f2c73451196a44fcc9faf826f" is_pre_election: true
I20260619 02:29:33.442837 19967 raft_consensus.cc:2454] T 8f671dce5d464b7e91d8a81836b668ec P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8223df6185ea409ebbdd26b192a64500 in term 0.
I20260619 02:29:33.442883 19687 raft_consensus.cc:2454] T 8f671dce5d464b7e91d8a81836b668ec P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8223df6185ea409ebbdd26b192a64500 in term 0.
I20260619 02:29:33.443231 19761 leader_election.cc:302] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5657d994279149cfac5b1b909f4f5b63, 8223df6185ea409ebbdd26b192a64500; no voters:
I20260619 02:29:33.443446 20044 raft_consensus.cc:2781] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:29:33.443538 20044 raft_consensus.cc:482] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:29:33.443598 20044 raft_consensus.cc:3037] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:33.444662 20044 raft_consensus.cc:504] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:33.445127 20044 leader_election.cc:288] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500 [CANDIDATE]: Term 1 election: Requested vote from peers cffd4b4f2c73451196a44fcc9faf826f (127.17.96.193:45915), 5657d994279149cfac5b1b909f4f5b63 (127.17.96.195:38173)
I20260619 02:29:33.445593 19967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f671dce5d464b7e91d8a81836b668ec" candidate_uuid: "8223df6185ea409ebbdd26b192a64500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5657d994279149cfac5b1b909f4f5b63"
I20260619 02:29:33.445585 19687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f671dce5d464b7e91d8a81836b668ec" candidate_uuid: "8223df6185ea409ebbdd26b192a64500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cffd4b4f2c73451196a44fcc9faf826f"
I20260619 02:29:33.445793 19967 raft_consensus.cc:3037] T 8f671dce5d464b7e91d8a81836b668ec P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:33.445792 19687 raft_consensus.cc:3037] T 8f671dce5d464b7e91d8a81836b668ec P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:33.446820 19967 raft_consensus.cc:2454] T 8f671dce5d464b7e91d8a81836b668ec P 5657d994279149cfac5b1b909f4f5b63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8223df6185ea409ebbdd26b192a64500 in term 1.
I20260619 02:29:33.447115 19687 raft_consensus.cc:2454] T 8f671dce5d464b7e91d8a81836b668ec P cffd4b4f2c73451196a44fcc9faf826f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8223df6185ea409ebbdd26b192a64500 in term 1.
I20260619 02:29:33.447227 19761 leader_election.cc:302] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5657d994279149cfac5b1b909f4f5b63, 8223df6185ea409ebbdd26b192a64500; no voters:
I20260619 02:29:33.447644 20044 raft_consensus.cc:2781] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:33.447850 20044 raft_consensus.cc:686] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500 [term 1 LEADER]: Becoming Leader. State: Replica: 8223df6185ea409ebbdd26b192a64500, State: Running, Role: LEADER
I20260619 02:29:33.448020 20044 consensus_queue.cc:237] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:33.450107 19527 catalog_manager.cc:5495] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8223df6185ea409ebbdd26b192a64500 (127.17.96.194). New cstate: current_term: 1 leader_uuid: "8223df6185ea409ebbdd26b192a64500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } health_report { overall_health: UNKNOWN } } }
I20260619 02:29:33.486549 19687 raft_consensus.cc:1261] T 8f671dce5d464b7e91d8a81836b668ec P cffd4b4f2c73451196a44fcc9faf826f [term 1 FOLLOWER]: Refusing update from remote peer 8223df6185ea409ebbdd26b192a64500: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:29:33.487188 19966 raft_consensus.cc:1261] T 8f671dce5d464b7e91d8a81836b668ec P 5657d994279149cfac5b1b909f4f5b63 [term 1 FOLLOWER]: Refusing update from remote peer 8223df6185ea409ebbdd26b192a64500: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:29:33.487926 20047 consensus_queue.cc:1035] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:29:33.488600 20047 consensus_queue.cc:1035] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 02:29:33.509974 20064 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 02:29:35.081601 19527 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:29:35.081825 19527 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:29:35.084484 19527 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714: Sending DeleteTablet for 3 replicas of tablet 8f671dce5d464b7e91d8a81836b668ec
I20260619 02:29:35.085280 19947 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f671dce5d464b7e91d8a81836b668ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:35 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:59350
I20260619 02:29:35.085326 19667 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f671dce5d464b7e91d8a81836b668ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:35 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:53142
I20260619 02:29:35.085443 19807 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f671dce5d464b7e91d8a81836b668ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:35 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:41856
I20260619 02:29:35.085851 20111 tablet_replica.cc:330] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500: stopping tablet replica
I20260619 02:29:35.086018 20111 raft_consensus.cc:2229] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:29:35.086026 20110 tablet_replica.cc:330] T 8f671dce5d464b7e91d8a81836b668ec P cffd4b4f2c73451196a44fcc9faf826f: stopping tablet replica
I20260619 02:29:35.086158 20109 tablet_replica.cc:330] T 8f671dce5d464b7e91d8a81836b668ec P 5657d994279149cfac5b1b909f4f5b63: stopping tablet replica
I20260619 02:29:35.086289 20111 raft_consensus.cc:2258] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:35.086516 20110 raft_consensus.cc:2229] T 8f671dce5d464b7e91d8a81836b668ec P cffd4b4f2c73451196a44fcc9faf826f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:29:35.086668 20110 raft_consensus.cc:2258] T 8f671dce5d464b7e91d8a81836b668ec P cffd4b4f2c73451196a44fcc9faf826f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:35.086336 20109 raft_consensus.cc:2229] T 8f671dce5d464b7e91d8a81836b668ec P 5657d994279149cfac5b1b909f4f5b63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:29:35.086912 20109 raft_consensus.cc:2258] T 8f671dce5d464b7e91d8a81836b668ec P 5657d994279149cfac5b1b909f4f5b63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:35.087853 20110 ts_tablet_manager.cc:1905] T 8f671dce5d464b7e91d8a81836b668ec P cffd4b4f2c73451196a44fcc9faf826f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:35.087917 20109 ts_tablet_manager.cc:1905] T 8f671dce5d464b7e91d8a81836b668ec P 5657d994279149cfac5b1b909f4f5b63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:35.089672 20111 ts_tablet_manager.cc:1905] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:35.091012 20109 ts_tablet_manager.cc:1918] T 8f671dce5d464b7e91d8a81836b668ec P 5657d994279149cfac5b1b909f4f5b63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:29:35.091132 20109 log.cc:1196] T 8f671dce5d464b7e91d8a81836b668ec P 5657d994279149cfac5b1b909f4f5b63: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/8f671dce5d464b7e91d8a81836b668ec
I20260619 02:29:35.091226 20110 ts_tablet_manager.cc:1918] T 8f671dce5d464b7e91d8a81836b668ec P cffd4b4f2c73451196a44fcc9faf826f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:29:35.091337 20110 log.cc:1196] T 8f671dce5d464b7e91d8a81836b668ec P cffd4b4f2c73451196a44fcc9faf826f: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-0/wal/wals/8f671dce5d464b7e91d8a81836b668ec
I20260619 02:29:35.091511 20109 ts_tablet_manager.cc:1939] T 8f671dce5d464b7e91d8a81836b668ec P 5657d994279149cfac5b1b909f4f5b63: Deleting consensus metadata
I20260619 02:29:35.091729 20110 ts_tablet_manager.cc:1939] T 8f671dce5d464b7e91d8a81836b668ec P cffd4b4f2c73451196a44fcc9faf826f: Deleting consensus metadata
I20260619 02:29:35.092388 19517 catalog_manager.cc:4841] TS 5657d994279149cfac5b1b909f4f5b63 (127.17.96.195:38173): tablet 8f671dce5d464b7e91d8a81836b668ec (table test-table [id=eb08b9102b4749f49ce798cf88f9916c]) successfully deleted
I20260619 02:29:35.092554 19516 catalog_manager.cc:4841] TS cffd4b4f2c73451196a44fcc9faf826f (127.17.96.193:45915): tablet 8f671dce5d464b7e91d8a81836b668ec (table test-table [id=eb08b9102b4749f49ce798cf88f9916c]) successfully deleted
I20260619 02:29:35.093029 20111 ts_tablet_manager.cc:1918] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:29:35.093137 20111 log.cc:1196] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/8f671dce5d464b7e91d8a81836b668ec
I20260619 02:29:35.093521 20111 ts_tablet_manager.cc:1939] T 8f671dce5d464b7e91d8a81836b668ec P 8223df6185ea409ebbdd26b192a64500: Deleting consensus metadata
I20260619 02:29:35.094468 19516 catalog_manager.cc:4841] TS 8223df6185ea409ebbdd26b192a64500 (127.17.96.194:41639): tablet 8f671dce5d464b7e91d8a81836b668ec (table test-table [id=eb08b9102b4749f49ce798cf88f9916c]) successfully deleted
Jun 19 02:29:35 dist-test-slave-4qw6 krb5kdc[19485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836175, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 02:29:35 dist-test-slave-4qw6 krb5kdc[19485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836175, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:35.123345 20118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:35.111833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50636 (local address 127.17.96.254:42351)
0619 02:29:35.112190 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:35.112199 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:35.112419 (+ 220us) server_negotiation.cc:407] Connection header received
0619 02:29:35.112511 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:35.112514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:35.112579 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:35.112689 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:35.113581 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:35.114252 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:35.115156 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:35.115416 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:35.115520 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:35.119448 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:35.119451 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:35.119454 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:35.119489 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:35.121157 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:35.121742 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:35.121820 (+ 78us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:35.121826 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:35.121957 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:35.122408 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:35.122411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:35.122413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:35.122661 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:35.122791 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:35.123044 (+ 253us) server_negotiation.cc:299] Negotiation successful
0619 02:29:35.123179 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":146,"threads_started":1}
I20260619 02:29:35.126602 19527 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50636:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:29:35.127167 19527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:29:35.132706 19667 tablet_service.cc:1467] Processing CreateTablet for tablet cd8dece850df4f1eab8c655b3a47017c (DEFAULT_TABLE table=test-table [id=ce8a2d5871bb40d0b44a8f5d28af6bb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:35.133155 19667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd8dece850df4f1eab8c655b3a47017c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:35.134160 19807 tablet_service.cc:1467] Processing CreateTablet for tablet cd8dece850df4f1eab8c655b3a47017c (DEFAULT_TABLE table=test-table [id=ce8a2d5871bb40d0b44a8f5d28af6bb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:35.134311 19947 tablet_service.cc:1467] Processing CreateTablet for tablet cd8dece850df4f1eab8c655b3a47017c (DEFAULT_TABLE table=test-table [id=ce8a2d5871bb40d0b44a8f5d28af6bb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:35.134514 19807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd8dece850df4f1eab8c655b3a47017c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:35.134755 19947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd8dece850df4f1eab8c655b3a47017c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:35.136397 20120 tablet_bootstrap.cc:492] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f: Bootstrap starting.
I20260619 02:29:35.137706 20121 tablet_bootstrap.cc:492] T cd8dece850df4f1eab8c655b3a47017c P 8223df6185ea409ebbdd26b192a64500: Bootstrap starting.
I20260619 02:29:35.138000 20122 tablet_bootstrap.cc:492] T cd8dece850df4f1eab8c655b3a47017c P 5657d994279149cfac5b1b909f4f5b63: Bootstrap starting.
I20260619 02:29:35.138389 20120 tablet_bootstrap.cc:654] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:35.139078 20121 tablet_bootstrap.cc:654] T cd8dece850df4f1eab8c655b3a47017c P 8223df6185ea409ebbdd26b192a64500: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:35.139744 20122 tablet_bootstrap.cc:654] T cd8dece850df4f1eab8c655b3a47017c P 5657d994279149cfac5b1b909f4f5b63: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:35.140924 20120 tablet_bootstrap.cc:492] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f: No bootstrap required, opened a new log
I20260619 02:29:35.141048 20120 ts_tablet_manager.cc:1397] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 02:29:35.141192 20121 tablet_bootstrap.cc:492] T cd8dece850df4f1eab8c655b3a47017c P 8223df6185ea409ebbdd26b192a64500: No bootstrap required, opened a new log
I20260619 02:29:35.141319 20122 tablet_bootstrap.cc:492] T cd8dece850df4f1eab8c655b3a47017c P 5657d994279149cfac5b1b909f4f5b63: No bootstrap required, opened a new log
I20260619 02:29:35.141340 20121 ts_tablet_manager.cc:1397] T cd8dece850df4f1eab8c655b3a47017c P 8223df6185ea409ebbdd26b192a64500: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260619 02:29:35.141418 20122 ts_tablet_manager.cc:1397] T cd8dece850df4f1eab8c655b3a47017c P 5657d994279149cfac5b1b909f4f5b63: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260619 02:29:35.141690 20120 raft_consensus.cc:348] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:35.141885 20120 raft_consensus.cc:374] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:35.141937 20120 raft_consensus.cc:729] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cffd4b4f2c73451196a44fcc9faf826f, State: Initialized, Role: FOLLOWER
I20260619 02:29:35.141964 20122 raft_consensus.cc:348] T cd8dece850df4f1eab8c655b3a47017c P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:35.141942 20121 raft_consensus.cc:348] T cd8dece850df4f1eab8c655b3a47017c P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:35.142086 20122 raft_consensus.cc:374] T cd8dece850df4f1eab8c655b3a47017c P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:35.142118 20122 raft_consensus.cc:729] T cd8dece850df4f1eab8c655b3a47017c P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5657d994279149cfac5b1b909f4f5b63, State: Initialized, Role: FOLLOWER
I20260619 02:29:35.142105 20120 consensus_queue.cc:260] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:35.142100 20121 raft_consensus.cc:374] T cd8dece850df4f1eab8c655b3a47017c P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:35.142295 20122 consensus_queue.cc:260] T cd8dece850df4f1eab8c655b3a47017c P 5657d994279149cfac5b1b909f4f5b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:35.142398 20121 raft_consensus.cc:729] T cd8dece850df4f1eab8c655b3a47017c P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8223df6185ea409ebbdd26b192a64500, State: Initialized, Role: FOLLOWER
I20260619 02:29:35.142608 20120 ts_tablet_manager.cc:1428] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:29:35.142691 20122 ts_tablet_manager.cc:1428] T cd8dece850df4f1eab8c655b3a47017c P 5657d994279149cfac5b1b909f4f5b63: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260619 02:29:35.143146 20121 consensus_queue.cc:260] T cd8dece850df4f1eab8c655b3a47017c P 8223df6185ea409ebbdd26b192a64500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:35.144465 20121 ts_tablet_manager.cc:1428] T cd8dece850df4f1eab8c655b3a47017c P 8223df6185ea409ebbdd26b192a64500: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 02:29:35.302009 20126 raft_consensus.cc:482] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:29:35.302176 20126 raft_consensus.cc:504] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:35.303443 20126 leader_election.cc:288] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8223df6185ea409ebbdd26b192a64500 (127.17.96.194:41639), 5657d994279149cfac5b1b909f4f5b63 (127.17.96.195:38173)
I20260619 02:29:35.310535 20131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:35.303898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:35127 (local address 127.17.96.195:38173)
0619 02:29:35.304214 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:35.304223 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:35.304331 (+ 108us) server_negotiation.cc:407] Connection header received
0619 02:29:35.304527 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:35.304531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:35.304596 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:35.304703 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:35.305192 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:35.306138 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:35.308668 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:35.309461 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:35.309652 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:35.309716 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:35.310212 (+ 496us) server_negotiation.cc:299] Negotiation successful
0619 02:29:35.310338 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":93,"threads_started":1}
I20260619 02:29:35.310580 20129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:35.303829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:55243 (local address 127.17.96.194:41639)
0619 02:29:35.304177 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:35.304191 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:35.304416 (+ 225us) server_negotiation.cc:407] Connection header received
0619 02:29:35.304539 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:35.304542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:35.304596 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:35.304703 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:35.305192 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:35.306138 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:35.308668 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:35.309537 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:35.309660 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:35.309715 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:35.310291 (+ 576us) server_negotiation.cc:299] Negotiation successful
0619 02:29:35.310434 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":124,"threads_started":1}
I20260619 02:29:35.310539 20128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:35.303716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:38173 (local address 127.17.96.193:35127)
0619 02:29:35.304210 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:35.304231 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:35.304379 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:35.304730 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:35.304733 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:35.304777 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:35.305065 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:35.305073 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:35.306381 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:35.306387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:35.308446 (+ 2059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:35.308473 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:35.309687 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:35.309692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:35.309979 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:35.310046 (+ 67us) client_negotiation.cc:769] Sending connection context
0619 02:29:35.310152 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 02:29:35.310282 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":428,"spinlock_wait_cycles":4992,"thread_start_us":161,"threads_started":1}
I20260619 02:29:35.310556 20130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:35.303719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:41639 (local address 127.17.96.193:55243)
0619 02:29:35.304322 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:35.304336 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:35.304434 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:35.304730 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:35.304733 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:35.304776 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:35.305066 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:35.305073 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:35.306381 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:35.306387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:35.308446 (+ 2059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:35.308473 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:35.309687 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:35.309692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:35.309978 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:35.310046 (+ 68us) client_negotiation.cc:769] Sending connection context
0619 02:29:35.310249 (+ 203us) client_negotiation.cc:240] Negotiation successful
0619 02:29:35.310351 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":433,"spinlock_wait_cycles":31744,"thread_start_us":119,"threads_started":1}
I20260619 02:29:35.311450 19826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd8dece850df4f1eab8c655b3a47017c" candidate_uuid: "cffd4b4f2c73451196a44fcc9faf826f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8223df6185ea409ebbdd26b192a64500" is_pre_election: true
I20260619 02:29:35.311619 19967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd8dece850df4f1eab8c655b3a47017c" candidate_uuid: "cffd4b4f2c73451196a44fcc9faf826f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5657d994279149cfac5b1b909f4f5b63" is_pre_election: true
I20260619 02:29:35.311837 19967 raft_consensus.cc:2454] T cd8dece850df4f1eab8c655b3a47017c P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cffd4b4f2c73451196a44fcc9faf826f in term 0.
I20260619 02:29:35.311849 19826 raft_consensus.cc:2454] T cd8dece850df4f1eab8c655b3a47017c P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cffd4b4f2c73451196a44fcc9faf826f in term 0.
I20260619 02:29:35.312470 19621 leader_election.cc:302] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5657d994279149cfac5b1b909f4f5b63, 8223df6185ea409ebbdd26b192a64500, cffd4b4f2c73451196a44fcc9faf826f; no voters:
I20260619 02:29:35.312860 20126 raft_consensus.cc:2781] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:29:35.312933 20126 raft_consensus.cc:482] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:29:35.312976 20126 raft_consensus.cc:3037] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:35.314410 20126 raft_consensus.cc:504] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:35.314860 20126 leader_election.cc:288] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f [CANDIDATE]: Term 1 election: Requested vote from peers 8223df6185ea409ebbdd26b192a64500 (127.17.96.194:41639), 5657d994279149cfac5b1b909f4f5b63 (127.17.96.195:38173)
I20260619 02:29:35.315411 19967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd8dece850df4f1eab8c655b3a47017c" candidate_uuid: "cffd4b4f2c73451196a44fcc9faf826f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5657d994279149cfac5b1b909f4f5b63"
I20260619 02:29:35.315546 19967 raft_consensus.cc:3037] T cd8dece850df4f1eab8c655b3a47017c P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:35.315760 19826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd8dece850df4f1eab8c655b3a47017c" candidate_uuid: "cffd4b4f2c73451196a44fcc9faf826f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8223df6185ea409ebbdd26b192a64500"
I20260619 02:29:35.315932 19826 raft_consensus.cc:3037] T cd8dece850df4f1eab8c655b3a47017c P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:35.317016 19826 raft_consensus.cc:2454] T cd8dece850df4f1eab8c655b3a47017c P 8223df6185ea409ebbdd26b192a64500 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cffd4b4f2c73451196a44fcc9faf826f in term 1.
I20260619 02:29:35.317023 19967 raft_consensus.cc:2454] T cd8dece850df4f1eab8c655b3a47017c P 5657d994279149cfac5b1b909f4f5b63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cffd4b4f2c73451196a44fcc9faf826f in term 1.
I20260619 02:29:35.317456 19621 leader_election.cc:302] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5657d994279149cfac5b1b909f4f5b63, 8223df6185ea409ebbdd26b192a64500, cffd4b4f2c73451196a44fcc9faf826f; no voters:
I20260619 02:29:35.317780 20126 raft_consensus.cc:2781] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:35.318048 20126 raft_consensus.cc:686] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f [term 1 LEADER]: Becoming Leader. State: Replica: cffd4b4f2c73451196a44fcc9faf826f, State: Running, Role: LEADER
I20260619 02:29:35.318368 20126 consensus_queue.cc:237] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:35.321030 19527 catalog_manager.cc:5495] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f reported cstate change: term changed from 0 to 1, leader changed from <none> to cffd4b4f2c73451196a44fcc9faf826f (127.17.96.193). New cstate: current_term: 1 leader_uuid: "cffd4b4f2c73451196a44fcc9faf826f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } health_report { overall_health: UNKNOWN } } }
I20260619 02:29:35.351270 20135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:35.347033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53152 (local address 127.17.96.193:45915)
0619 02:29:35.347330 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:35.347340 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:35.347355 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:29:35.347445 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:35.347448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:35.347507 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:35.347646 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:29:35.347936 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:35.348557 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:35.349496 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:35.349828 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:35.349966 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:35.350277 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:29:35.350796 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:29:35.350930 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:35.351013 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 02:29:35.351080 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":112,"threads_started":1}
I20260619 02:29:35.364549 20129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:35.360376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:51737 (local address 127.17.96.194:41639)
0619 02:29:35.360509 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:35.360513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:35.360656 (+ 143us) server_negotiation.cc:407] Connection header received
0619 02:29:35.360774 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:35.360776 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:35.360823 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:35.360915 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:29:35.361230 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:35.361842 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:35.362949 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:35.363258 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:35.363410 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:35.363977 (+ 567us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:29:35.364073 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:29:35.364158 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:35.364341 (+ 183us) server_negotiation.cc:299] Negotiation successful
0619 02:29:35.364416 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 02:29:35.364586 20138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:35.360229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:41639 (local address 127.17.96.193:51737)
0619 02:29:35.360574 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:35.360589 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:35.360694 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:35.360974 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:35.360977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:35.360986 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:29:35.361126 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:35.361131 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:35.361997 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:35.362000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:35.362731 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:35.362741 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:35.363532 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:35.363537 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:35.363673 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:35.363751 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:29:35.364232 (+ 481us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:29:35.364241 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 02:29:35.364315 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 02:29:35.364414 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":143,"threads_started":1}
I20260619 02:29:35.372038 19967 raft_consensus.cc:1261] T cd8dece850df4f1eab8c655b3a47017c P 5657d994279149cfac5b1b909f4f5b63 [term 1 FOLLOWER]: Refusing update from remote peer cffd4b4f2c73451196a44fcc9faf826f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:29:35.372334 19826 raft_consensus.cc:1261] T cd8dece850df4f1eab8c655b3a47017c P 8223df6185ea409ebbdd26b192a64500 [term 1 FOLLOWER]: Refusing update from remote peer cffd4b4f2c73451196a44fcc9faf826f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:29:35.372757 20132 consensus_queue.cc:1035] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:29:35.373009 20126 consensus_queue.cc:1035] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:29:35.394487 20135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:35.390715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:33101 (local address 127.17.96.193:45915)
0619 02:29:35.390867 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:35.390872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:35.390887 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:29:35.390948 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:35.390951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:35.391011 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:35.391109 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:29:35.391389 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:35.391969 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:35.392886 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:35.393187 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:35.393310 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:35.393823 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:29:35.393965 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:29:35.394065 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:35.394217 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 02:29:35.394327 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:29:35.394678 20142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:35.390162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:45915 (local address 127.17.96.194:33101)
0619 02:29:35.390495 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:35.390515 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:35.390664 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:35.391119 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:35.391123 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:35.391132 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:29:35.391271 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:35.391278 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:35.392080 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:35.392086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:35.392761 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:35.392769 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:35.393337 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:35.393341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:35.393493 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:35.393561 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:29:35.394096 (+ 535us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:29:35.394103 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 02:29:35.394188 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 02:29:35.394428 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":106,"threads_started":1}
I20260619 02:29:35.422792 19527 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:29:35.423081 19527 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:29:35.426095 19527 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714: Sending DeleteTablet for 3 replicas of tablet cd8dece850df4f1eab8c655b3a47017c
I20260619 02:29:35.426925 19947 tablet_service.cc:1514] Processing DeleteTablet for tablet cd8dece850df4f1eab8c655b3a47017c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:35 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:59350
I20260619 02:29:35.426955 19667 tablet_service.cc:1514] Processing DeleteTablet for tablet cd8dece850df4f1eab8c655b3a47017c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:35 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:53142
I20260619 02:29:35.427173 20109 tablet_replica.cc:330] T cd8dece850df4f1eab8c655b3a47017c P 5657d994279149cfac5b1b909f4f5b63: stopping tablet replica
I20260619 02:29:35.427193 20110 tablet_replica.cc:330] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f: stopping tablet replica
I20260619 02:29:35.427314 20109 raft_consensus.cc:2229] T cd8dece850df4f1eab8c655b3a47017c P 5657d994279149cfac5b1b909f4f5b63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:29:35.427325 20110 raft_consensus.cc:2229] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:29:35.427482 20109 raft_consensus.cc:2258] T cd8dece850df4f1eab8c655b3a47017c P 5657d994279149cfac5b1b909f4f5b63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:35.427706 20110 raft_consensus.cc:2258] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:35.427722 19527 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50636:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:29:35.428246 19527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:29:35.428601 20109 ts_tablet_manager.cc:1905] T cd8dece850df4f1eab8c655b3a47017c P 5657d994279149cfac5b1b909f4f5b63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:35.428922 20110 ts_tablet_manager.cc:1905] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:35.429111 19807 tablet_service.cc:1514] Processing DeleteTablet for tablet cd8dece850df4f1eab8c655b3a47017c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:35 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:41856
I20260619 02:29:35.429315 20111 tablet_replica.cc:330] T cd8dece850df4f1eab8c655b3a47017c P 8223df6185ea409ebbdd26b192a64500: stopping tablet replica
I20260619 02:29:35.429425 20111 raft_consensus.cc:2229] T cd8dece850df4f1eab8c655b3a47017c P 8223df6185ea409ebbdd26b192a64500 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:29:35.429541 20111 raft_consensus.cc:2258] T cd8dece850df4f1eab8c655b3a47017c P 8223df6185ea409ebbdd26b192a64500 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:35.430416 20111 ts_tablet_manager.cc:1905] T cd8dece850df4f1eab8c655b3a47017c P 8223df6185ea409ebbdd26b192a64500: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:35.431793 20109 ts_tablet_manager.cc:1918] T cd8dece850df4f1eab8c655b3a47017c P 5657d994279149cfac5b1b909f4f5b63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:29:35.431877 20109 log.cc:1196] T cd8dece850df4f1eab8c655b3a47017c P 5657d994279149cfac5b1b909f4f5b63: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/cd8dece850df4f1eab8c655b3a47017c
I20260619 02:29:35.431888 20110 ts_tablet_manager.cc:1918] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:29:35.431978 20110 log.cc:1196] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-0/wal/wals/cd8dece850df4f1eab8c655b3a47017c
I20260619 02:29:35.432181 20109 ts_tablet_manager.cc:1939] T cd8dece850df4f1eab8c655b3a47017c P 5657d994279149cfac5b1b909f4f5b63: Deleting consensus metadata
I20260619 02:29:35.432278 20110 ts_tablet_manager.cc:1939] T cd8dece850df4f1eab8c655b3a47017c P cffd4b4f2c73451196a44fcc9faf826f: Deleting consensus metadata
I20260619 02:29:35.433257 19517 catalog_manager.cc:4841] TS 5657d994279149cfac5b1b909f4f5b63 (127.17.96.195:38173): tablet cd8dece850df4f1eab8c655b3a47017c (table test-table [id=ce8a2d5871bb40d0b44a8f5d28af6bb0]) successfully deleted
I20260619 02:29:35.433360 19516 catalog_manager.cc:4841] TS cffd4b4f2c73451196a44fcc9faf826f (127.17.96.193:45915): tablet cd8dece850df4f1eab8c655b3a47017c (table test-table [id=ce8a2d5871bb40d0b44a8f5d28af6bb0]) successfully deleted
I20260619 02:29:35.433495 20111 ts_tablet_manager.cc:1918] T cd8dece850df4f1eab8c655b3a47017c P 8223df6185ea409ebbdd26b192a64500: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:29:35.433583 20111 log.cc:1196] T cd8dece850df4f1eab8c655b3a47017c P 8223df6185ea409ebbdd26b192a64500: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/cd8dece850df4f1eab8c655b3a47017c
I20260619 02:29:35.433887 20111 ts_tablet_manager.cc:1939] T cd8dece850df4f1eab8c655b3a47017c P 8223df6185ea409ebbdd26b192a64500: Deleting consensus metadata
I20260619 02:29:35.434649 19516 catalog_manager.cc:4841] TS 8223df6185ea409ebbdd26b192a64500 (127.17.96.194:41639): tablet cd8dece850df4f1eab8c655b3a47017c (table test-table [id=ce8a2d5871bb40d0b44a8f5d28af6bb0]) successfully deleted
I20260619 02:29:35.438169 19667 tablet_service.cc:1467] Processing CreateTablet for tablet da7403e770924c6f9d988ecf1d493503 (DEFAULT_TABLE table=test-table [id=895890c11f6c4016b6bbc227868e7f3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:35.438217 19807 tablet_service.cc:1467] Processing CreateTablet for tablet da7403e770924c6f9d988ecf1d493503 (DEFAULT_TABLE table=test-table [id=895890c11f6c4016b6bbc227868e7f3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:35.438421 19947 tablet_service.cc:1467] Processing CreateTablet for tablet da7403e770924c6f9d988ecf1d493503 (DEFAULT_TABLE table=test-table [id=895890c11f6c4016b6bbc227868e7f3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:35.438488 19667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da7403e770924c6f9d988ecf1d493503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:35.438519 19807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da7403e770924c6f9d988ecf1d493503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:35.438680 19947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da7403e770924c6f9d988ecf1d493503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:35.441123 20120 tablet_bootstrap.cc:492] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f: Bootstrap starting.
I20260619 02:29:35.441443 20122 tablet_bootstrap.cc:492] T da7403e770924c6f9d988ecf1d493503 P 5657d994279149cfac5b1b909f4f5b63: Bootstrap starting.
I20260619 02:29:35.442287 20120 tablet_bootstrap.cc:654] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:35.442472 20122 tablet_bootstrap.cc:654] T da7403e770924c6f9d988ecf1d493503 P 5657d994279149cfac5b1b909f4f5b63: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:35.443773 20120 tablet_bootstrap.cc:492] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f: No bootstrap required, opened a new log
I20260619 02:29:35.443854 20120 ts_tablet_manager.cc:1397] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 02:29:35.444331 20120 raft_consensus.cc:348] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:35.444465 20120 raft_consensus.cc:374] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:35.444515 20120 raft_consensus.cc:729] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cffd4b4f2c73451196a44fcc9faf826f, State: Initialized, Role: FOLLOWER
I20260619 02:29:35.444655 20120 consensus_queue.cc:260] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:35.444972 20120 ts_tablet_manager.cc:1428] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 02:29:35.445501 20121 tablet_bootstrap.cc:492] T da7403e770924c6f9d988ecf1d493503 P 8223df6185ea409ebbdd26b192a64500: Bootstrap starting.
I20260619 02:29:35.446455 20122 tablet_bootstrap.cc:492] T da7403e770924c6f9d988ecf1d493503 P 5657d994279149cfac5b1b909f4f5b63: No bootstrap required, opened a new log
I20260619 02:29:35.446542 20121 tablet_bootstrap.cc:654] T da7403e770924c6f9d988ecf1d493503 P 8223df6185ea409ebbdd26b192a64500: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:35.446573 20122 ts_tablet_manager.cc:1397] T da7403e770924c6f9d988ecf1d493503 P 5657d994279149cfac5b1b909f4f5b63: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260619 02:29:35.447140 20122 raft_consensus.cc:348] T da7403e770924c6f9d988ecf1d493503 P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:35.447299 20122 raft_consensus.cc:374] T da7403e770924c6f9d988ecf1d493503 P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:35.447345 20122 raft_consensus.cc:729] T da7403e770924c6f9d988ecf1d493503 P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5657d994279149cfac5b1b909f4f5b63, State: Initialized, Role: FOLLOWER
I20260619 02:29:35.447515 20122 consensus_queue.cc:260] T da7403e770924c6f9d988ecf1d493503 P 5657d994279149cfac5b1b909f4f5b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:35.447929 20122 ts_tablet_manager.cc:1428] T da7403e770924c6f9d988ecf1d493503 P 5657d994279149cfac5b1b909f4f5b63: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260619 02:29:35.448081 20121 tablet_bootstrap.cc:492] T da7403e770924c6f9d988ecf1d493503 P 8223df6185ea409ebbdd26b192a64500: No bootstrap required, opened a new log
I20260619 02:29:35.448179 20121 ts_tablet_manager.cc:1397] T da7403e770924c6f9d988ecf1d493503 P 8223df6185ea409ebbdd26b192a64500: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 02:29:35.448625 20121 raft_consensus.cc:348] T da7403e770924c6f9d988ecf1d493503 P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:35.448760 20121 raft_consensus.cc:374] T da7403e770924c6f9d988ecf1d493503 P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:35.448803 20121 raft_consensus.cc:729] T da7403e770924c6f9d988ecf1d493503 P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8223df6185ea409ebbdd26b192a64500, State: Initialized, Role: FOLLOWER
I20260619 02:29:35.448962 20121 consensus_queue.cc:260] T da7403e770924c6f9d988ecf1d493503 P 8223df6185ea409ebbdd26b192a64500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:35.449311 20121 ts_tablet_manager.cc:1428] T da7403e770924c6f9d988ecf1d493503 P 8223df6185ea409ebbdd26b192a64500: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260619 02:29:35.642583 20143 raft_consensus.cc:482] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:29:35.642812 20143 raft_consensus.cc:504] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:35.643342 20143 leader_election.cc:288] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8223df6185ea409ebbdd26b192a64500 (127.17.96.194:41639), 5657d994279149cfac5b1b909f4f5b63 (127.17.96.195:38173)
I20260619 02:29:35.643915 19826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da7403e770924c6f9d988ecf1d493503" candidate_uuid: "cffd4b4f2c73451196a44fcc9faf826f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8223df6185ea409ebbdd26b192a64500" is_pre_election: true
I20260619 02:29:35.643915 19967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da7403e770924c6f9d988ecf1d493503" candidate_uuid: "cffd4b4f2c73451196a44fcc9faf826f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5657d994279149cfac5b1b909f4f5b63" is_pre_election: true
I20260619 02:29:35.644102 19967 raft_consensus.cc:2454] T da7403e770924c6f9d988ecf1d493503 P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cffd4b4f2c73451196a44fcc9faf826f in term 0.
I20260619 02:29:35.644102 19826 raft_consensus.cc:2454] T da7403e770924c6f9d988ecf1d493503 P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cffd4b4f2c73451196a44fcc9faf826f in term 0.
I20260619 02:29:35.644503 19621 leader_election.cc:302] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5657d994279149cfac5b1b909f4f5b63, cffd4b4f2c73451196a44fcc9faf826f; no voters:
I20260619 02:29:35.644752 20143 raft_consensus.cc:2781] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:29:35.644855 20143 raft_consensus.cc:482] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:29:35.644915 20143 raft_consensus.cc:3037] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:35.646113 20143 raft_consensus.cc:504] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:35.646524 20143 leader_election.cc:288] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f [CANDIDATE]: Term 1 election: Requested vote from peers 8223df6185ea409ebbdd26b192a64500 (127.17.96.194:41639), 5657d994279149cfac5b1b909f4f5b63 (127.17.96.195:38173)
I20260619 02:29:35.647158 19967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da7403e770924c6f9d988ecf1d493503" candidate_uuid: "cffd4b4f2c73451196a44fcc9faf826f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5657d994279149cfac5b1b909f4f5b63"
I20260619 02:29:35.647158 19826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da7403e770924c6f9d988ecf1d493503" candidate_uuid: "cffd4b4f2c73451196a44fcc9faf826f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8223df6185ea409ebbdd26b192a64500"
I20260619 02:29:35.647324 19826 raft_consensus.cc:3037] T da7403e770924c6f9d988ecf1d493503 P 8223df6185ea409ebbdd26b192a64500 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:35.647324 19967 raft_consensus.cc:3037] T da7403e770924c6f9d988ecf1d493503 P 5657d994279149cfac5b1b909f4f5b63 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:35.648557 19967 raft_consensus.cc:2454] T da7403e770924c6f9d988ecf1d493503 P 5657d994279149cfac5b1b909f4f5b63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cffd4b4f2c73451196a44fcc9faf826f in term 1.
I20260619 02:29:35.648557 19826 raft_consensus.cc:2454] T da7403e770924c6f9d988ecf1d493503 P 8223df6185ea409ebbdd26b192a64500 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cffd4b4f2c73451196a44fcc9faf826f in term 1.
I20260619 02:29:35.648953 19620 leader_election.cc:302] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8223df6185ea409ebbdd26b192a64500, cffd4b4f2c73451196a44fcc9faf826f; no voters:
I20260619 02:29:35.649170 20143 raft_consensus.cc:2781] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:35.649277 20143 raft_consensus.cc:686] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f [term 1 LEADER]: Becoming Leader. State: Replica: cffd4b4f2c73451196a44fcc9faf826f, State: Running, Role: LEADER
I20260619 02:29:35.649446 20143 consensus_queue.cc:237] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } }
I20260619 02:29:35.651278 19528 catalog_manager.cc:5495] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f reported cstate change: term changed from 0 to 1, leader changed from <none> to cffd4b4f2c73451196a44fcc9faf826f (127.17.96.193). New cstate: current_term: 1 leader_uuid: "cffd4b4f2c73451196a44fcc9faf826f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cffd4b4f2c73451196a44fcc9faf826f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 45915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 } health_report { overall_health: UNKNOWN } } }
I20260619 02:29:35.700887 19826 raft_consensus.cc:1261] T da7403e770924c6f9d988ecf1d493503 P 8223df6185ea409ebbdd26b192a64500 [term 1 FOLLOWER]: Refusing update from remote peer cffd4b4f2c73451196a44fcc9faf826f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:29:35.701035 19967 raft_consensus.cc:1261] T da7403e770924c6f9d988ecf1d493503 P 5657d994279149cfac5b1b909f4f5b63 [term 1 FOLLOWER]: Refusing update from remote peer cffd4b4f2c73451196a44fcc9faf826f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:29:35.701392 20126 consensus_queue.cc:1035] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8223df6185ea409ebbdd26b192a64500" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:29:35.701586 20143 consensus_queue.cc:1035] T da7403e770924c6f9d988ecf1d493503 P cffd4b4f2c73451196a44fcc9faf826f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5657d994279149cfac5b1b909f4f5b63" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 02:29:35.716610 20094 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 02:29:37.278370 19525 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:29:37.278600 19525 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:29:37.281419 19525 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f20885720a046c2898e54b1f82ed714: Sending DeleteTablet for 3 replicas of tablet da7403e770924c6f9d988ecf1d493503
I20260619 02:29:37.282330 19807 tablet_service.cc:1514] Processing DeleteTablet for tablet da7403e770924c6f9d988ecf1d493503 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:37 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:41856
I20260619 02:29:37.282325 19667 tablet_service.cc:1514] Processing DeleteTablet for tablet da7403e770924c6f9d988ecf1d493503 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:37 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:53142
I20260619 02:29:37.282450 19947 tablet_service.cc:1514] Processing DeleteTablet for tablet da7403e770924c6f9d988ecf1d493503 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:37 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:59350
I20260619 02:29:37.282876 20174 tablet_replica.cc:330] T da7403e770924c6f9d988ecf1d493503 P 8223df6185ea409ebbdd26b192a64500: stopping tablet replica
I20260619 02:29:37.282917 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 19602
I20260619 02:29:37.284710 20176 tablet_replica.cc:330] T da7403e770924c6f9d988ecf1d493503 P 5657d994279149cfac5b1b909f4f5b63: stopping tablet replica
I20260619 02:29:37.284744 20174 raft_consensus.cc:2229] T da7403e770924c6f9d988ecf1d493503 P 8223df6185ea409ebbdd26b192a64500 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:29:37.284896 20176 raft_consensus.cc:2229] T da7403e770924c6f9d988ecf1d493503 P 5657d994279149cfac5b1b909f4f5b63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:29:37.284937 20174 raft_consensus.cc:2258] T da7403e770924c6f9d988ecf1d493503 P 8223df6185ea409ebbdd26b192a64500 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:37.285061 20176 raft_consensus.cc:2258] T da7403e770924c6f9d988ecf1d493503 P 5657d994279149cfac5b1b909f4f5b63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:37.285857 20176 ts_tablet_manager.cc:1905] T da7403e770924c6f9d988ecf1d493503 P 5657d994279149cfac5b1b909f4f5b63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:37.288275 20176 ts_tablet_manager.cc:1918] T da7403e770924c6f9d988ecf1d493503 P 5657d994279149cfac5b1b909f4f5b63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:29:37.288353 20176 log.cc:1196] T da7403e770924c6f9d988ecf1d493503 P 5657d994279149cfac5b1b909f4f5b63: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/da7403e770924c6f9d988ecf1d493503
I20260619 02:29:37.288627 20176 ts_tablet_manager.cc:1939] T da7403e770924c6f9d988ecf1d493503 P 5657d994279149cfac5b1b909f4f5b63: Deleting consensus metadata
I20260619 02:29:37.289500 19517 catalog_manager.cc:4841] TS 5657d994279149cfac5b1b909f4f5b63 (127.17.96.195:38173): tablet da7403e770924c6f9d988ecf1d493503 (table test-table [id=895890c11f6c4016b6bbc227868e7f3e]) successfully deleted
I20260619 02:29:37.290246 20174 ts_tablet_manager.cc:1905] T da7403e770924c6f9d988ecf1d493503 P 8223df6185ea409ebbdd26b192a64500: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 02:29:37.291558 19516 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.96.193:45915 (error 108)
I20260619 02:29:37.292884 20178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:37.292393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:45915 (local address 127.0.0.1:53166)
0619 02:29:37.292649 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:37.292736 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.193:45915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":97,"threads_started":1}
I20260619 02:29:37.293005 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 19742
I20260619 02:29:37.293095 20174 ts_tablet_manager.cc:1918] T da7403e770924c6f9d988ecf1d493503 P 8223df6185ea409ebbdd26b192a64500: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
W20260619 02:29:37.293197 19516 catalog_manager.cc:4584] TS cffd4b4f2c73451196a44fcc9faf826f (127.17.96.193:45915): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet da7403e770924c6f9d988ecf1d493503: Network error: Client connection negotiation failed: client connection to 127.17.96.193:45915: connect: Connection refused (error 111)
I20260619 02:29:37.293257 20174 log.cc:1196] T da7403e770924c6f9d988ecf1d493503 P 8223df6185ea409ebbdd26b192a64500: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/da7403e770924c6f9d988ecf1d493503
I20260619 02:29:37.300823 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 19882
I20260619 02:29:37.302585 20178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:37.302126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:41639 (local address 127.0.0.1:41884)
0619 02:29:37.302317 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:37.302403 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.194:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260619 02:29:37.308363 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 19498
2026-06-19T02:29:37Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (7193 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:29:37 dist-test-slave-4qw6 krb5kdc[20181](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:29:37 dist-test-slave-4qw6 krb5kdc[20181](info): set up 2 sockets
Jun 19 02:29:37 dist-test-slave-4qw6 krb5kdc[20181](info): commencing operation
krb5kdc: starting...
W20260619 02:29:38.454456 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:29:38 dist-test-slave-4qw6 krb5kdc[20181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:29:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:29:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:29:38.596511 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:33373
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:40511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:33373
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:38.729296 20194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:38.729674 20194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:38.729728 20194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:38.734997 20194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:29:38.735113 20194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:38.735136 20194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:38.735157 20194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:29:38.735174 20194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:29:38.742734 20194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:40511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:33373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:33373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:38.744190 20194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:38.745422 20194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:38.755651 20200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:38.755657 20199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:38.755725 20194 server_base.cc:1034] running on GCE node
W20260619 02:29:38.755651 20202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:38.756726 20194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:38.758122 20194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:38.759372 20194 hybrid_clock.cc:648] HybridClock initialized: now 1781836178759341 us; error 44 us; skew 500 ppm
Jun 19 02:29:38 dist-test-slave-4qw6 krb5kdc[20181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836178, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:38.762724 20194 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:29:38.763796 20194 webserver.cc:460] Webserver started at http://127.17.96.254:35885/ using document root <none> and password file <none>
I20260619 02:29:38.764433 20194 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:38.764531 20194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:38.764789 20194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:38.767201 20194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "9394e6fda95f42f3b1ad1a5d15a1fb58"
format_stamp: "Formatted at 2026-06-19 02:29:38 on dist-test-slave-4qw6"
server_key: "7bc54ce21329f22bcb7828376424fb1d"
server_key_iv: "6a40dc853579c079fd06f0902856992c"
server_key_version: "encryptionkey@0"
I20260619 02:29:38.767872 20194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "9394e6fda95f42f3b1ad1a5d15a1fb58"
format_stamp: "Formatted at 2026-06-19 02:29:38 on dist-test-slave-4qw6"
server_key: "7bc54ce21329f22bcb7828376424fb1d"
server_key_iv: "6a40dc853579c079fd06f0902856992c"
server_key_version: "encryptionkey@0"
I20260619 02:29:38.772348 20194 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:29:38.775285 20209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:38.776803 20194 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:29:38.776973 20194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "9394e6fda95f42f3b1ad1a5d15a1fb58"
format_stamp: "Formatted at 2026-06-19 02:29:38 on dist-test-slave-4qw6"
server_key: "7bc54ce21329f22bcb7828376424fb1d"
server_key_iv: "6a40dc853579c079fd06f0902856992c"
server_key_version: "encryptionkey@0"
I20260619 02:29:38.777117 20194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:38.803778 20194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:38.807543 20194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:38.807806 20194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:38.816344 20194 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:33373
I20260619 02:29:38.816351 20261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:33373 every 8 connection(s)
I20260619 02:29:38.817498 20194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:29:38.820839 20262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:38.825712 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 20194
I20260619 02:29:38.825868 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:29:38.826169 17795 external_mini_cluster.cc:1428] Setting key 51ef66c83903d801e152021d4e0ed137
I20260619 02:29:38.828171 20262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58: Bootstrap starting.
I20260619 02:29:38.831123 20262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:38.832261 20262 log.cc:824] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:38.835484 20262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58: No bootstrap required, opened a new log
Jun 19 02:29:38 dist-test-slave-4qw6 krb5kdc[20181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:38.840456 20262 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9394e6fda95f42f3b1ad1a5d15a1fb58" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33373 } }
I20260619 02:29:38.840718 20262 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:38.840802 20262 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9394e6fda95f42f3b1ad1a5d15a1fb58, State: Initialized, Role: FOLLOWER
I20260619 02:29:38.841315 20262 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9394e6fda95f42f3b1ad1a5d15a1fb58" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33373 } }
I20260619 02:29:38.841476 20262 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:29:38.841614 20262 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:29:38.841738 20262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:38.842885 20262 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9394e6fda95f42f3b1ad1a5d15a1fb58" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33373 } }
I20260619 02:29:38.843284 20262 leader_election.cc:302] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9394e6fda95f42f3b1ad1a5d15a1fb58; no voters:
I20260619 02:29:38.843675 20262 leader_election.cc:288] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:29:38.843809 20267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:38.844051 20267 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58 [term 1 LEADER]: Becoming Leader. State: Replica: 9394e6fda95f42f3b1ad1a5d15a1fb58, State: Running, Role: LEADER
I20260619 02:29:38.844221 20265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:38.828277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53654 (local address 127.17.96.254:33373)
0619 02:29:38.828889 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:38.828909 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:38.828961 (+ 52us) server_negotiation.cc:407] Connection header received
0619 02:29:38.829954 (+ 993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:38.829980 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:38.830462 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:38.830890 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:38.832103 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:38.833382 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:38.834830 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:38.835384 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:38.835525 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:38.838663 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:38.838669 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:38.838688 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:38.838723 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:38.840535 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:38.841093 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:38.841099 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:38.841104 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:38.841226 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:38.841557 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:38.841562 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:38.841568 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:38.842150 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:38.842380 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:38.842709 (+ 329us) server_negotiation.cc:299] Negotiation successful
0619 02:29:38.843105 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":199,"threads_started":1}
I20260619 02:29:38.844472 20267 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9394e6fda95f42f3b1ad1a5d15a1fb58" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33373 } }
I20260619 02:29:38.845105 20262 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:29:38.847673 20268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9394e6fda95f42f3b1ad1a5d15a1fb58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9394e6fda95f42f3b1ad1a5d15a1fb58" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33373 } } }
I20260619 02:29:38.848078 20268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:38.847817 20269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9394e6fda95f42f3b1ad1a5d15a1fb58. Latest consensus state: current_term: 1 leader_uuid: "9394e6fda95f42f3b1ad1a5d15a1fb58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9394e6fda95f42f3b1ad1a5d15a1fb58" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33373 } } }
I20260619 02:29:38.848186 20269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58 [sys.catalog]: This master's current role is: LEADER
W20260619 02:29:38.850562 20281 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:29:38.850672 20281 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:29:38.850747 20282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:29:38.851641 20282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:29:38.858038 20282 catalog_manager.cc:1347] Generated new cluster ID: 2f3daa2f9f1e437a9d0fcf24423d59eb
I20260619 02:29:38.858129 20282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:29:38.876174 20282 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:29:38.877381 20282 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:29:38.887893 20282 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9394e6fda95f42f3b1ad1a5d15a1fb58: Generated new TSK 0
I20260619 02:29:38.888787 20282 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.96.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
I20260619 02:29:38.921705 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:33373
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:40511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:39.050351 20288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:39.050732 20288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:39.050814 20288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:39.055692 20288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:39.055809 20288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:39.055930 20288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:29:39.062170 20288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:40511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:33373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:39.063411 20288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:39.064569 20288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:39.076802 20293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:39.076787 20294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:39.076794 20296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:39.077292 20288 server_base.cc:1034] running on GCE node
I20260619 02:29:39.078032 20288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:39.078969 20288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:39.080214 20288 hybrid_clock.cc:648] HybridClock initialized: now 1781836179080161 us; error 71 us; skew 500 ppm
Jun 19 02:29:39 dist-test-slave-4qw6 krb5kdc[20181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836179, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:39.083827 20288 init.cc:377] Logged in from keytab as kudu/127.17.96.193@KRBTEST.COM (short username kudu)
I20260619 02:29:39.085076 20288 webserver.cc:460] Webserver started at http://127.17.96.193:42113/ using document root <none> and password file <none>
I20260619 02:29:39.085986 20288 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:39.086081 20288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:39.086330 20288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:39.089422 20288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "6a9aeb4b6bb243238c91afd8f805ff09"
format_stamp: "Formatted at 2026-06-19 02:29:39 on dist-test-slave-4qw6"
server_key: "d815ee1fa3129e60db4b77a13a725710"
server_key_iv: "74b94d4420c4d4fad6a87f9a95701a9a"
server_key_version: "encryptionkey@0"
I20260619 02:29:39.090217 20288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "6a9aeb4b6bb243238c91afd8f805ff09"
format_stamp: "Formatted at 2026-06-19 02:29:39 on dist-test-slave-4qw6"
server_key: "d815ee1fa3129e60db4b77a13a725710"
server_key_iv: "74b94d4420c4d4fad6a87f9a95701a9a"
server_key_version: "encryptionkey@0"
I20260619 02:29:39.095266 20288 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 02:29:39.098820 20303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:39.100473 20288 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:29:39.100622 20288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "6a9aeb4b6bb243238c91afd8f805ff09"
format_stamp: "Formatted at 2026-06-19 02:29:39 on dist-test-slave-4qw6"
server_key: "d815ee1fa3129e60db4b77a13a725710"
server_key_iv: "74b94d4420c4d4fad6a87f9a95701a9a"
server_key_version: "encryptionkey@0"
I20260619 02:29:39.100800 20288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:39.112484 20288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:39.116169 20288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:39.116397 20288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:39.117066 20288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:39.118111 20288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:39.118170 20288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:39.118216 20288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:39.118273 20288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:39.129244 20288 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:36173
I20260619 02:29:39.129262 20416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:36173 every 8 connection(s)
I20260619 02:29:39.130357 20288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:29:39.140272 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 20288
I20260619 02:29:39.140388 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:29:39.140707 17795 external_mini_cluster.cc:1428] Setting key f23fc4358938b44af1615d8b10587d3a
Jun 19 02:29:39 dist-test-slave-4qw6 krb5kdc[20181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836179, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:39.146944 20265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:39.132460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:49523 (local address 127.17.96.254:33373)
0619 02:29:39.132625 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:39.132629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:39.133658 (+ 1029us) server_negotiation.cc:407] Connection header received
0619 02:29:39.135300 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:39.135303 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:39.135373 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:39.135495 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:39.136920 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:39.137496 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:39.138848 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:39.139105 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:39.139232 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:39.142259 (+ 3027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:39.142262 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:39.142265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:39.142290 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:39.144114 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:39.144860 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:39.144866 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:39.144882 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:39.144993 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:39.145370 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:39.145373 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:39.145377 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:39.145681 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:39.145851 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:39.146456 (+ 605us) server_negotiation.cc:299] Negotiation successful
0619 02:29:39.146647 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 02:29:39.147840 20419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:39.132900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33373 (local address 127.17.96.193:49523)
0619 02:29:39.133485 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:39.133538 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:39.135043 (+ 1505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:39.135699 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:39.135707 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:39.136151 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:39.136729 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:39.136744 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:39.137652 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:39.137656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:39.138671 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:39.138683 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:39.139288 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:39.139292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:39.139498 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:39.140221 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:39.140241 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:39.142062 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:39.144285 (+ 2223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:39.144291 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:39.144309 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:39.144732 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:39.145092 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:39.145094 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:39.145095 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:39.145257 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:39.145851 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:39.145857 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:39.146156 (+ 299us) client_negotiation.cc:769] Sending connection context
0619 02:29:39.146344 (+ 188us) client_negotiation.cc:240] Negotiation successful
0619 02:29:39.146672 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":142,"threads_started":1}
I20260619 02:29:39.149324 20417 heartbeater.cc:347] Connected to a master server at 127.17.96.254:33373
I20260619 02:29:39.149664 20417 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:39.150465 20417 heartbeater.cc:511] Master 127.17.96.254:33373 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.96.194@KRBTEST.COM; defaulting to no policy
I20260619 02:29:39.152489 20219 ts_manager.cc:194] Registered new tserver with Master: 6a9aeb4b6bb243238c91afd8f805ff09 (127.17.96.193:36173)
I20260619 02:29:39.153971 20219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:49523
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
I20260619 02:29:39.173575 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:33373
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:40511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:39.303699 20422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:39.304087 20422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:39.304140 20422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:39.309063 20422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:39.309175 20422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:39.309253 20422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:29:39.316718 20422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:40511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:33373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:39.318397 20422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:39.319677 20422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:39.331490 20430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:39.331506 20427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:39.331511 20428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:39.331902 20422 server_base.cc:1034] running on GCE node
I20260619 02:29:39.332489 20422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:39.333433 20422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:39.334787 20422 hybrid_clock.cc:648] HybridClock initialized: now 1781836179334707 us; error 101 us; skew 500 ppm
Jun 19 02:29:39 dist-test-slave-4qw6 krb5kdc[20181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836179, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:39.338727 20422 init.cc:377] Logged in from keytab as kudu/127.17.96.194@KRBTEST.COM (short username kudu)
I20260619 02:29:39.340116 20422 webserver.cc:460] Webserver started at http://127.17.96.194:45505/ using document root <none> and password file <none>
I20260619 02:29:39.340869 20422 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:39.340946 20422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:39.341243 20422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:39.344043 20422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "441a2251b24a400aa86f122c5d25f6f6"
format_stamp: "Formatted at 2026-06-19 02:29:39 on dist-test-slave-4qw6"
server_key: "9a9b6ef16a27b5a8a46cfe1fb14be294"
server_key_iv: "35e6899fa72a64f680e6f5c621054c99"
server_key_version: "encryptionkey@0"
I20260619 02:29:39.344723 20422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "441a2251b24a400aa86f122c5d25f6f6"
format_stamp: "Formatted at 2026-06-19 02:29:39 on dist-test-slave-4qw6"
server_key: "9a9b6ef16a27b5a8a46cfe1fb14be294"
server_key_iv: "35e6899fa72a64f680e6f5c621054c99"
server_key_version: "encryptionkey@0"
I20260619 02:29:39.350715 20422 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260619 02:29:39.354589 20437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:39.356271 20422 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:29:39.356433 20422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "441a2251b24a400aa86f122c5d25f6f6"
format_stamp: "Formatted at 2026-06-19 02:29:39 on dist-test-slave-4qw6"
server_key: "9a9b6ef16a27b5a8a46cfe1fb14be294"
server_key_iv: "35e6899fa72a64f680e6f5c621054c99"
server_key_version: "encryptionkey@0"
I20260619 02:29:39.356566 20422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:39.372277 20422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:39.376168 20422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:39.376367 20422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:39.376976 20422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:39.378031 20422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:39.378091 20422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:39.378137 20422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:39.378199 20422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:39.389065 20422 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:46209
I20260619 02:29:39.389086 20550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:46209 every 8 connection(s)
I20260619 02:29:39.390199 20422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:29:39.392182 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 20422
I20260619 02:29:39.392290 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:29:39.392601 17795 external_mini_cluster.cc:1428] Setting key b0b144db400d9f828e46d4359b61c8be
Jun 19 02:29:39 dist-test-slave-4qw6 krb5kdc[20181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836179, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.96.195@KRBTEST.COM; defaulting to no policy
I20260619 02:29:39.407932 20265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:39.392360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:32915 (local address 127.17.96.254:33373)
0619 02:29:39.392528 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:39.392534 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:39.393645 (+ 1111us) server_negotiation.cc:407] Connection header received
0619 02:29:39.395589 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:39.395593 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:39.395664 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:39.395778 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:39.397534 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:39.398215 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:39.399608 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:39.399875 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:39.400016 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:39.403304 (+ 3288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:39.403309 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:39.403311 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:39.403343 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:39.405349 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:39.406081 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:39.406086 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:39.406088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:39.406185 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:39.406628 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:39.406632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:39.406634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:39.406864 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:39.406997 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:39.407583 (+ 586us) server_negotiation.cc:299] Negotiation successful
0619 02:29:39.407741 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 02:29:39.408876 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:39.392854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33373 (local address 127.17.96.194:32915)
0619 02:29:39.393456 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:39.393519 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:39.395255 (+ 1736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:39.396035 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:39.396048 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:39.396687 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:39.397328 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:39.397345 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:39.398409 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:39.398413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:39.399436 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:39.399449 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:39.400064 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:39.400068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:39.400291 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:39.401055 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:39.401091 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:39.403049 (+ 1958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:39.405509 (+ 2460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:39.405516 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:39.405539 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:39.405957 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:39.406320 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:39.406326 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:39.406328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:39.406510 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:39.406994 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:39.407002 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:39.407305 (+ 303us) client_negotiation.cc:769] Sending connection context
0619 02:29:39.407519 (+ 214us) client_negotiation.cc:240] Negotiation successful
0619 02:29:39.407929 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":159,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.195@KRBTEST.COM" created.
I20260619 02:29:39.410259 20551 heartbeater.cc:347] Connected to a master server at 127.17.96.254:33373
I20260619 02:29:39.410567 20551 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:39.411283 20551 heartbeater.cc:511] Master 127.17.96.254:33373 requested a full tablet report, sending...
I20260619 02:29:39.412587 20219 ts_manager.cc:194] Registered new tserver with Master: 441a2251b24a400aa86f122c5d25f6f6 (127.17.96.194:46209)
I20260619 02:29:39.413255 20219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:32915
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
I20260619 02:29:39.429250 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:33373
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:40511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:39.551604 20556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:39.551956 20556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:39.552075 20556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:39.556392 20556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:39.556468 20556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:39.556529 20556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:29:39.562302 20556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:40511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:33373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:39.563433 20556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:39.564417 20556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:39.574445 20564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:39.574431 20561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:39.574512 20556 server_base.cc:1034] running on GCE node
W20260619 02:29:39.574438 20562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:39.575028 20556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:39.575743 20556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:39.576933 20556 hybrid_clock.cc:648] HybridClock initialized: now 1781836179576919 us; error 36 us; skew 500 ppm
Jun 19 02:29:39 dist-test-slave-4qw6 krb5kdc[20181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836179, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:39.579747 20556 init.cc:377] Logged in from keytab as kudu/127.17.96.195@KRBTEST.COM (short username kudu)
I20260619 02:29:39.580713 20556 webserver.cc:460] Webserver started at http://127.17.96.195:41993/ using document root <none> and password file <none>
I20260619 02:29:39.581300 20556 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:39.581357 20556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:39.581532 20556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:39.583745 20556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "fe0ab8fe48e44fb5ba85b0491a0117ec"
format_stamp: "Formatted at 2026-06-19 02:29:39 on dist-test-slave-4qw6"
server_key: "dc3e9820cccec32e3025ec1f7d071452"
server_key_iv: "679964e8a6ef75b97a19d6f0065936ef"
server_key_version: "encryptionkey@0"
I20260619 02:29:39.584285 20556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "fe0ab8fe48e44fb5ba85b0491a0117ec"
format_stamp: "Formatted at 2026-06-19 02:29:39 on dist-test-slave-4qw6"
server_key: "dc3e9820cccec32e3025ec1f7d071452"
server_key_iv: "679964e8a6ef75b97a19d6f0065936ef"
server_key_version: "encryptionkey@0"
I20260619 02:29:39.588168 20556 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:29:39.590607 20571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:39.591872 20556 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 02:29:39.591975 20556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "fe0ab8fe48e44fb5ba85b0491a0117ec"
format_stamp: "Formatted at 2026-06-19 02:29:39 on dist-test-slave-4qw6"
server_key: "dc3e9820cccec32e3025ec1f7d071452"
server_key_iv: "679964e8a6ef75b97a19d6f0065936ef"
server_key_version: "encryptionkey@0"
I20260619 02:29:39.592073 20556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:39.620913 20556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:39.624396 20556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:39.624564 20556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:39.625146 20556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:39.625991 20556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:39.626040 20556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:39.626081 20556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:39.626097 20556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:39.635793 20556 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:45239
I20260619 02:29:39.635811 20684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:45239 every 8 connection(s)
I20260619 02:29:39.636854 20556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:29:39.646970 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 20556
I20260619 02:29:39.647156 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:29:39.647500 17795 external_mini_cluster.cc:1428] Setting key f614b20ae6e4e9041a0fc635572d3e78
Jun 19 02:29:39 dist-test-slave-4qw6 krb5kdc[20181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836179, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:39.653457 20265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:39.638849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:52093 (local address 127.17.96.254:33373)
0619 02:29:39.639017 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:39.639021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:39.639933 (+ 912us) server_negotiation.cc:407] Connection header received
0619 02:29:39.641492 (+ 1559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:39.641498 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:39.641583 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:39.641711 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:39.643193 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:39.643799 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:39.645283 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:39.645546 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:39.645653 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:39.649208 (+ 3555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:39.649212 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:39.649215 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:39.649263 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:39.651169 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:39.651767 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:39.651769 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:39.651771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:39.651854 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:39.652222 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:39.652224 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:39.652225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:39.652429 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:39.652524 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:39.653152 (+ 628us) server_negotiation.cc:299] Negotiation successful
0619 02:29:39.653295 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 02:29:39.654445 20687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:39.639283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33373 (local address 127.17.96.195:52093)
0619 02:29:39.639761 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:39.639807 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:39.641081 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:39.641966 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:39.641975 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:39.642405 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:39.642946 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:39.642962 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:39.643978 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:39.643981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:39.645031 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:39.645043 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:39.645716 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:39.645719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:39.645958 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:39.646696 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:39.646715 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:39.648869 (+ 2154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:39.651313 (+ 2444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:39.651319 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:39.651335 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:39.651666 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:39.651954 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:39.651957 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:39.651959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:39.652121 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:39.652535 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:39.652540 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:39.652839 (+ 299us) client_negotiation.cc:769] Sending connection context
0619 02:29:39.653023 (+ 184us) client_negotiation.cc:240] Negotiation successful
0619 02:29:39.653335 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":134,"threads_started":1}
I20260619 02:29:39.656121 20685 heartbeater.cc:347] Connected to a master server at 127.17.96.254:33373
I20260619 02:29:39.656426 20685 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:39.656991 20685 heartbeater.cc:511] Master 127.17.96.254:33373 requested a full tablet report, sending...
I20260619 02:29:39.658179 20219 ts_manager.cc:194] Registered new tserver with Master: fe0ab8fe48e44fb5ba85b0491a0117ec (127.17.96.195:45239)
I20260619 02:29:39.658761 20219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:52093
I20260619 02:29:39.662802 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 02:29:39.678031 20265 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:29:39.676321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53660 (local address 127.17.96.254:33373)
0619 02:29:39.676506 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:39.676511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:39.676667 (+ 156us) server_negotiation.cc:407] Connection header received
0619 02:29:39.676862 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:39.676866 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:39.676925 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:39.677013 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:39.677858 (+ 845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53660: BlockingRecv error: recv got EOF from 127.0.0.1:53660 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:29:39.679399 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 20288
I20260619 02:29:39.685448 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 20422
I20260619 02:29:39.691828 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 20556
I20260619 02:29:39.698994 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 20194
2026-06-19T02:29:39Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2388 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:29:39 dist-test-slave-4qw6 krb5kdc[20697](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:29:39 dist-test-slave-4qw6 krb5kdc[20697](info): set up 2 sockets
Jun 19 02:29:39 dist-test-slave-4qw6 krb5kdc[20697](info): commencing operation
krb5kdc: starting...
W20260619 02:29:40.814791 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:29:40 dist-test-slave-4qw6 krb5kdc[20697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:29:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:29:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:29:40.951315 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:35083
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:33601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:35083
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:41.073310 20710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:41.073681 20710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:41.073729 20710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:41.078106 20710 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:29:41.078193 20710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:41.078217 20710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:41.078238 20710 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:29:41.078256 20710 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:29:41.083967 20710 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:33601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:35083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:35083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:41.085197 20710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:41.086269 20710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:41.095098 20715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:41.095140 20716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:41.095244 20718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:41.095853 20710 server_base.cc:1034] running on GCE node
I20260619 02:29:41.096599 20710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:41.098096 20710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:41.099267 20710 hybrid_clock.cc:648] HybridClock initialized: now 1781836181099254 us; error 33 us; skew 500 ppm
Jun 19 02:29:41 dist-test-slave-4qw6 krb5kdc[20697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836181, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:41.102155 20710 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:29:41.103111 20710 webserver.cc:460] Webserver started at http://127.17.96.254:32977/ using document root <none> and password file <none>
I20260619 02:29:41.103726 20710 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:41.103791 20710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:41.104022 20710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:41.106282 20710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "8755c9c4dac64349b486b396f4eaf983"
format_stamp: "Formatted at 2026-06-19 02:29:41 on dist-test-slave-4qw6"
server_key: "53f191ae3fe484021b37e716675e1f7b"
server_key_iv: "ef53cb7041076a5b4e0ea12aaa2f606e"
server_key_version: "encryptionkey@0"
I20260619 02:29:41.106892 20710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "8755c9c4dac64349b486b396f4eaf983"
format_stamp: "Formatted at 2026-06-19 02:29:41 on dist-test-slave-4qw6"
server_key: "53f191ae3fe484021b37e716675e1f7b"
server_key_iv: "ef53cb7041076a5b4e0ea12aaa2f606e"
server_key_version: "encryptionkey@0"
I20260619 02:29:41.110869 20710 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 02:29:41.113422 20725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:41.114784 20710 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 02:29:41.114931 20710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "8755c9c4dac64349b486b396f4eaf983"
format_stamp: "Formatted at 2026-06-19 02:29:41 on dist-test-slave-4qw6"
server_key: "53f191ae3fe484021b37e716675e1f7b"
server_key_iv: "ef53cb7041076a5b4e0ea12aaa2f606e"
server_key_version: "encryptionkey@0"
I20260619 02:29:41.115054 20710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:41.143327 20710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:41.146615 20710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:41.146863 20710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:41.154112 20710 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:35083
I20260619 02:29:41.154409 20777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:35083 every 8 connection(s)
I20260619 02:29:41.155205 20710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:29:41.158151 20778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:41.158612 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 20710
I20260619 02:29:41.158757 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:29:41.159029 17795 external_mini_cluster.cc:1428] Setting key 79dbbb8415ceae28311dcd3c4d743551
I20260619 02:29:41.165720 20778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8755c9c4dac64349b486b396f4eaf983: Bootstrap starting.
I20260619 02:29:41.168557 20778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8755c9c4dac64349b486b396f4eaf983: Neither blocks nor log segments found. Creating new log.
Jun 19 02:29:41 dist-test-slave-4qw6 krb5kdc[20697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:41.169492 20778 log.cc:824] T 00000000000000000000000000000000 P 8755c9c4dac64349b486b396f4eaf983: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:41.171898 20778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8755c9c4dac64349b486b396f4eaf983: No bootstrap required, opened a new log
I20260619 02:29:41.174394 20781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:41.160432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47616 (local address 127.17.96.254:35083)
0619 02:29:41.160885 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:41.160911 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:41.160956 (+ 45us) server_negotiation.cc:407] Connection header received
0619 02:29:41.161711 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:41.161736 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:41.162217 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:41.162606 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:41.163320 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:41.164832 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:41.165697 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:41.166389 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:41.166545 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:41.169302 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:41.169308 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:41.169323 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:41.169353 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:41.171278 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:41.171812 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:41.171816 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:41.171821 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:41.171929 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:41.172296 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:41.172299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:41.172300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:41.172709 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:41.172854 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:41.173247 (+ 393us) server_negotiation.cc:299] Negotiation successful
0619 02:29:41.173570 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":116,"threads_started":1}
I20260619 02:29:41.176237 20778 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8755c9c4dac64349b486b396f4eaf983 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8755c9c4dac64349b486b396f4eaf983" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35083 } }
I20260619 02:29:41.176546 20778 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8755c9c4dac64349b486b396f4eaf983 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:41.176635 20778 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8755c9c4dac64349b486b396f4eaf983 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8755c9c4dac64349b486b396f4eaf983, State: Initialized, Role: FOLLOWER
I20260619 02:29:41.177160 20778 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8755c9c4dac64349b486b396f4eaf983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8755c9c4dac64349b486b396f4eaf983" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35083 } }
I20260619 02:29:41.177323 20778 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8755c9c4dac64349b486b396f4eaf983 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:29:41.177392 20778 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8755c9c4dac64349b486b396f4eaf983 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:29:41.177496 20778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8755c9c4dac64349b486b396f4eaf983 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:41.177601 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:29:41.178542 20778 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8755c9c4dac64349b486b396f4eaf983 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8755c9c4dac64349b486b396f4eaf983" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35083 } }
I20260619 02:29:41.178910 20778 leader_election.cc:302] T 00000000000000000000000000000000 P 8755c9c4dac64349b486b396f4eaf983 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8755c9c4dac64349b486b396f4eaf983; no voters:
I20260619 02:29:41.179266 20778 leader_election.cc:288] T 00000000000000000000000000000000 P 8755c9c4dac64349b486b396f4eaf983 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:29:41.179360 20783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8755c9c4dac64349b486b396f4eaf983 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:41.179538 20783 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8755c9c4dac64349b486b396f4eaf983 [term 1 LEADER]: Becoming Leader. State: Replica: 8755c9c4dac64349b486b396f4eaf983, State: Running, Role: LEADER
I20260619 02:29:41.179847 20783 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8755c9c4dac64349b486b396f4eaf983 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8755c9c4dac64349b486b396f4eaf983" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35083 } }
I20260619 02:29:41.180652 20778 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8755c9c4dac64349b486b396f4eaf983 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:29:41.181090 20783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8755c9c4dac64349b486b396f4eaf983 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8755c9c4dac64349b486b396f4eaf983. Latest consensus state: current_term: 1 leader_uuid: "8755c9c4dac64349b486b396f4eaf983" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8755c9c4dac64349b486b396f4eaf983" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35083 } } }
I20260619 02:29:41.181226 20783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8755c9c4dac64349b486b396f4eaf983 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:41.181798 20785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8755c9c4dac64349b486b396f4eaf983 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8755c9c4dac64349b486b396f4eaf983" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8755c9c4dac64349b486b396f4eaf983" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35083 } } }
I20260619 02:29:41.181977 20785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8755c9c4dac64349b486b396f4eaf983 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:41.182307 20788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:29:41.185966 20788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:29:41.192543 20788 catalog_manager.cc:1347] Generated new cluster ID: 93f0600594e74f0f82d2dc1433b5bf2d
I20260619 02:29:41.192651 20788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:29:41.203130 20781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:41.196975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47620 (local address 127.17.96.254:35083)
0619 02:29:41.197090 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:41.197095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:41.197381 (+ 286us) server_negotiation.cc:407] Connection header received
0619 02:29:41.197500 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:41.197504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:41.197568 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:41.197664 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:41.198286 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:41.199140 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:41.200308 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:41.200762 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:41.200869 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:41.201938 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:41.201942 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:41.201945 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:29:41.202590 (+ 645us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:41.202632 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:29:41.202654 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:41.202760 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:41.202887 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 02:29:41.202959 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":39}
I20260619 02:29:41.204917 20788 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:29:41.205950 20788 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:29:41.212594 20788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8755c9c4dac64349b486b396f4eaf983: Generated new TSK 0
I20260619 02:29:41.213382 20788 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:29:41.219916 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 20710
2026-06-19T02:29:41Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1517 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:29:41 dist-test-slave-4qw6 krb5kdc[20811](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:29:41 dist-test-slave-4qw6 krb5kdc[20811](info): set up 2 sockets
Jun 19 02:29:41 dist-test-slave-4qw6 krb5kdc[20811](info): commencing operation
krb5kdc: starting...
W20260619 02:29:42.334259 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:29:42 dist-test-slave-4qw6 krb5kdc[20811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:29:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:29:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:29:42.477574 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:45763
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:33893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:45763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:42.609865 20824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:42.610236 20824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:42.610338 20824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:42.615621 20824 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:29:42.615777 20824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:42.615845 20824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:42.615897 20824 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:29:42.615937 20824 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:29:42.624231 20824 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:33893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:45763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:45763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:42.625732 20824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:42.627029 20824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:42.638147 20832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:42.638147 20830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:42.638187 20829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:42.638684 20824 server_base.cc:1034] running on GCE node
I20260619 02:29:42.639616 20824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:42.641175 20824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:42.642436 20824 hybrid_clock.cc:648] HybridClock initialized: now 1781836182642394 us; error 69 us; skew 500 ppm
Jun 19 02:29:42 dist-test-slave-4qw6 krb5kdc[20811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836182, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:42.649518 20824 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:29:42.650851 20824 webserver.cc:460] Webserver started at http://127.17.96.254:44261/ using document root <none> and password file <none>
I20260619 02:29:42.651554 20824 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:42.651659 20824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:42.651934 20824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:42.654402 20824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "645a0c017fbe41098656925560533741"
format_stamp: "Formatted at 2026-06-19 02:29:42 on dist-test-slave-4qw6"
server_key: "13204e476a4316604c300e2695d61f02"
server_key_iv: "199a98e3cfbb8a6be4689a54ca46a43e"
server_key_version: "encryptionkey@0"
I20260619 02:29:42.655071 20824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "645a0c017fbe41098656925560533741"
format_stamp: "Formatted at 2026-06-19 02:29:42 on dist-test-slave-4qw6"
server_key: "13204e476a4316604c300e2695d61f02"
server_key_iv: "199a98e3cfbb8a6be4689a54ca46a43e"
server_key_version: "encryptionkey@0"
I20260619 02:29:42.661283 20824 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260619 02:29:42.665215 20839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:42.667081 20824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:29:42.667297 20824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "645a0c017fbe41098656925560533741"
format_stamp: "Formatted at 2026-06-19 02:29:42 on dist-test-slave-4qw6"
server_key: "13204e476a4316604c300e2695d61f02"
server_key_iv: "199a98e3cfbb8a6be4689a54ca46a43e"
server_key_version: "encryptionkey@0"
I20260619 02:29:42.667459 20824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:42.684376 20824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:42.688542 20824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:42.688799 20824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:42.697083 20824 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:45763
I20260619 02:29:42.697403 20891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:45763 every 8 connection(s)
I20260619 02:29:42.698446 20824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:29:42.701954 20892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:42.706539 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 20824
I20260619 02:29:42.706693 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:29:42.707051 17795 external_mini_cluster.cc:1428] Setting key 390a646d40693c4a661a240cbffc3528
I20260619 02:29:42.709400 20892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741: Bootstrap starting.
I20260619 02:29:42.712196 20892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:42.713092 20892 log.cc:824] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:42.715855 20892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741: No bootstrap required, opened a new log
Jun 19 02:29:42 dist-test-slave-4qw6 krb5kdc[20811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:42.720430 20892 raft_consensus.cc:348] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "645a0c017fbe41098656925560533741" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45763 } }
I20260619 02:29:42.720815 20892 raft_consensus.cc:374] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:42.720908 20892 raft_consensus.cc:729] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 645a0c017fbe41098656925560533741, State: Initialized, Role: FOLLOWER
I20260619 02:29:42.721433 20892 consensus_queue.cc:260] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "645a0c017fbe41098656925560533741" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45763 } }
I20260619 02:29:42.721617 20892 raft_consensus.cc:388] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:29:42.721710 20892 raft_consensus.cc:482] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:29:42.721856 20892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:42.723260 20892 raft_consensus.cc:504] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "645a0c017fbe41098656925560533741" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45763 } }
I20260619 02:29:42.723698 20892 leader_election.cc:302] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 645a0c017fbe41098656925560533741; no voters:
I20260619 02:29:42.724112 20892 leader_election.cc:288] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:29:42.724463 20897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:42.724745 20897 raft_consensus.cc:686] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741 [term 1 LEADER]: Becoming Leader. State: Replica: 645a0c017fbe41098656925560533741, State: Running, Role: LEADER
I20260619 02:29:42.725159 20897 consensus_queue.cc:237] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "645a0c017fbe41098656925560533741" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45763 } }
I20260619 02:29:42.725600 20892 sys_catalog.cc:564] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:29:42.725600 20895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:42.708819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45550 (local address 127.17.96.254:45763)
0619 02:29:42.709407 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:42.709427 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:42.709475 (+ 48us) server_negotiation.cc:407] Connection header received
0619 02:29:42.710392 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:42.710415 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:42.710846 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:42.711260 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:42.712235 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:42.713495 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:42.714832 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:42.715412 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:42.715577 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:42.718993 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:42.718999 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:42.719018 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:42.719055 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:42.721738 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:42.722393 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:42.722399 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:42.722405 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:42.722556 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:42.722878 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:42.722881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:42.722882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:42.723416 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:42.723680 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:42.724087 (+ 407us) server_negotiation.cc:299] Negotiation successful
0619 02:29:42.724480 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":173,"threads_started":1}
I20260619 02:29:42.727087 20899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 645a0c017fbe41098656925560533741. Latest consensus state: current_term: 1 leader_uuid: "645a0c017fbe41098656925560533741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "645a0c017fbe41098656925560533741" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45763 } } }
I20260619 02:29:42.727226 20899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:42.727572 20898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "645a0c017fbe41098656925560533741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "645a0c017fbe41098656925560533741" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45763 } } }
I20260619 02:29:42.727658 20898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741 [sys.catalog]: This master's current role is: LEADER
W20260619 02:29:42.731047 20912 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:29:42.731169 20912 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:29:42.731305 20907 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:29:42.732128 20907 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:29:42.739462 20907 catalog_manager.cc:1347] Generated new cluster ID: 9b6012c69c8649dd9e9e9f0fcb358de2
I20260619 02:29:42.739579 20907 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:29:42.751536 20907 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:29:42.752662 20907 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:29:42.757516 20907 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 645a0c017fbe41098656925560533741: Generated new TSK 0
I20260619 02:29:42.758688 20907 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.96.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
I20260619 02:29:42.798766 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45763
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:33893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:42.931538 20918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:42.932060 20918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:42.932169 20918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:42.936939 20918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:42.937085 20918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:42.937211 20918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:29:42.943727 20918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:33893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:42.945019 20918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:42.946195 20918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:42.957178 20923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:42.957178 20924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:42.957180 20926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:42.957667 20918 server_base.cc:1034] running on GCE node
I20260619 02:29:42.958231 20918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:42.959017 20918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:42.960194 20918 hybrid_clock.cc:648] HybridClock initialized: now 1781836182960172 us; error 42 us; skew 500 ppm
Jun 19 02:29:42 dist-test-slave-4qw6 krb5kdc[20811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836182, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:42.963572 20918 init.cc:377] Logged in from keytab as kudu/127.17.96.193@KRBTEST.COM (short username kudu)
I20260619 02:29:42.964617 20918 webserver.cc:460] Webserver started at http://127.17.96.193:40627/ using document root <none> and password file <none>
I20260619 02:29:42.965260 20918 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:42.965337 20918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:42.965528 20918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:42.967971 20918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "8e232a8ac27849c58d861fee9c762e54"
format_stamp: "Formatted at 2026-06-19 02:29:42 on dist-test-slave-4qw6"
server_key: "3740b0cb583cf4bdda72b117ac59ee15"
server_key_iv: "1d296f81acd947b2e69df1d030729978"
server_key_version: "encryptionkey@0"
I20260619 02:29:42.968551 20918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "8e232a8ac27849c58d861fee9c762e54"
format_stamp: "Formatted at 2026-06-19 02:29:42 on dist-test-slave-4qw6"
server_key: "3740b0cb583cf4bdda72b117ac59ee15"
server_key_iv: "1d296f81acd947b2e69df1d030729978"
server_key_version: "encryptionkey@0"
I20260619 02:29:42.973014 20918 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260619 02:29:42.975800 20933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:42.977195 20918 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 02:29:42.977348 20918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "8e232a8ac27849c58d861fee9c762e54"
format_stamp: "Formatted at 2026-06-19 02:29:42 on dist-test-slave-4qw6"
server_key: "3740b0cb583cf4bdda72b117ac59ee15"
server_key_iv: "1d296f81acd947b2e69df1d030729978"
server_key_version: "encryptionkey@0"
I20260619 02:29:42.977475 20918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:43.007409 20918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:43.012081 20918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:43.012360 20918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:43.013103 20918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:43.014119 20918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:43.014200 20918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:43.014279 20918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:43.014330 20918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:43.025228 20918 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:40663
I20260619 02:29:43.025271 21046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:40663 every 8 connection(s)
I20260619 02:29:43.026487 20918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:29:43.027746 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 20918
I20260619 02:29:43.027897 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:29:43.028257 17795 external_mini_cluster.cc:1428] Setting key 1d6a9ae17216de97f0589b3d8673c43f
WARNING: no policy specified for kudu/127.17.96.194@KRBTEST.COM; defaulting to no policy
Jun 19 02:29:43 dist-test-slave-4qw6 krb5kdc[20811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836182, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.194@KRBTEST.COM" created.
I20260619 02:29:43.044709 20895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:43.029907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:34651 (local address 127.17.96.254:45763)
0619 02:29:43.030048 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:43.030053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:43.030129 (+ 76us) server_negotiation.cc:407] Connection header received
0619 02:29:43.031846 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:43.031851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:43.032099 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:43.032219 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:43.033684 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:43.034366 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:43.035770 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:43.036052 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:43.036192 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:43.039665 (+ 3473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:43.039669 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:43.039672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:43.039704 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:43.041990 (+ 2286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:43.042707 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:43.042712 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:43.042714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:43.042845 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:43.043301 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:43.043305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:43.043307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:43.043561 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:43.043684 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:43.044313 (+ 629us) server_negotiation.cc:299] Negotiation successful
0619 02:29:43.044489 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 02:29:43.045686 21049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:43.029323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45763 (local address 127.17.96.193:34651)
0619 02:29:43.029963 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:43.030035 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:43.031559 (+ 1524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:43.032444 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:43.032453 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:43.032905 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:43.033490 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:43.033507 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:43.034531 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:43.034536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:43.035597 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:43.035610 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:43.036249 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:43.036254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:43.036482 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:43.037277 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:43.037307 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:43.039313 (+ 2006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:43.042163 (+ 2850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:43.042177 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:43.042197 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:43.042563 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:43.042981 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:43.042984 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:43.042986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:43.043167 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:43.043697 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:43.043707 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:43.044023 (+ 316us) client_negotiation.cc:769] Sending connection context
0619 02:29:43.044263 (+ 240us) client_negotiation.cc:240] Negotiation successful
0619 02:29:43.044689 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":127,"threads_started":1}
I20260619 02:29:43.047456 21047 heartbeater.cc:347] Connected to a master server at 127.17.96.254:45763
I20260619 02:29:43.047758 21047 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:43.048377 21047 heartbeater.cc:511] Master 127.17.96.254:45763 requested a full tablet report, sending...
I20260619 02:29:43.050817 20856 ts_manager.cc:194] Registered new tserver with Master: 8e232a8ac27849c58d861fee9c762e54 (127.17.96.193:40663)
I20260619 02:29:43.052469 20856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:34651
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
I20260619 02:29:43.061708 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45763
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:33893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:43.199990 21052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:43.200335 21052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:43.200392 21052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:43.205041 21052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:43.205149 21052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:43.205222 21052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:29:43.211234 21052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:33893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:43.212479 21052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:43.213666 21052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:43.227299 21060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:43.227340 21057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:43.227558 21052 server_base.cc:1034] running on GCE node
W20260619 02:29:43.227690 21058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:43.228214 21052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:43.229091 21052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:43.230294 21052 hybrid_clock.cc:648] HybridClock initialized: now 1781836183230256 us; error 56 us; skew 500 ppm
Jun 19 02:29:43 dist-test-slave-4qw6 krb5kdc[20811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836183, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:43.239162 21052 init.cc:377] Logged in from keytab as kudu/127.17.96.194@KRBTEST.COM (short username kudu)
I20260619 02:29:43.240473 21052 webserver.cc:460] Webserver started at http://127.17.96.194:44495/ using document root <none> and password file <none>
I20260619 02:29:43.241194 21052 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:43.241312 21052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:43.241592 21052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:43.244191 21052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "208fee193dc6475b82bd435959b3a050"
format_stamp: "Formatted at 2026-06-19 02:29:43 on dist-test-slave-4qw6"
server_key: "124b3fcdad169030713462101bae4797"
server_key_iv: "c405f381af62f0f16738438cc15d9b6a"
server_key_version: "encryptionkey@0"
I20260619 02:29:43.245052 21052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "208fee193dc6475b82bd435959b3a050"
format_stamp: "Formatted at 2026-06-19 02:29:43 on dist-test-slave-4qw6"
server_key: "124b3fcdad169030713462101bae4797"
server_key_iv: "c405f381af62f0f16738438cc15d9b6a"
server_key_version: "encryptionkey@0"
I20260619 02:29:43.250033 21052 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 02:29:43.253556 21067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:43.255456 21052 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 02:29:43.255688 21052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "208fee193dc6475b82bd435959b3a050"
format_stamp: "Formatted at 2026-06-19 02:29:43 on dist-test-slave-4qw6"
server_key: "124b3fcdad169030713462101bae4797"
server_key_iv: "c405f381af62f0f16738438cc15d9b6a"
server_key_version: "encryptionkey@0"
I20260619 02:29:43.255858 21052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:43.280947 21052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:43.285117 21052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:43.285392 21052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:43.286185 21052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:43.287343 21052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:43.287428 21052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:43.287510 21052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:43.287564 21052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:43.298348 21052 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:42639
I20260619 02:29:43.298372 21180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:42639 every 8 connection(s)
I20260619 02:29:43.299441 21052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:29:43.301465 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 21052
I20260619 02:29:43.301632 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:29:43.302068 17795 external_mini_cluster.cc:1428] Setting key 386115e7873cba1a5b1e483a31846dbd
Jun 19 02:29:43 dist-test-slave-4qw6 krb5kdc[20811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836183, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.96.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.195@KRBTEST.COM" created.
I20260619 02:29:43.316567 20895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:43.301738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:47913 (local address 127.17.96.254:45763)
0619 02:29:43.302015 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:43.302021 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:43.303109 (+ 1088us) server_negotiation.cc:407] Connection header received
0619 02:29:43.304702 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:43.304707 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:43.304774 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:43.304871 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:43.306388 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:43.306999 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:43.308362 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:43.308632 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:43.308772 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:43.312135 (+ 3363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:43.312139 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:43.312142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:43.312173 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:43.314078 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:43.314757 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:43.314761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:43.314763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:43.314867 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:43.315258 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:43.315262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:43.315264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:43.315495 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:43.315616 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:43.316218 (+ 602us) server_negotiation.cc:299] Negotiation successful
0619 02:29:43.316378 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 02:29:43.317589 21183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:43.302325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45763 (local address 127.17.96.194:47913)
0619 02:29:43.302923 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:43.302987 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:43.304443 (+ 1456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:43.305078 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:43.305087 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:43.305536 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:43.306203 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:43.306222 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:43.307174 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:43.307178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:43.308197 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:43.308209 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:43.308835 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:43.308840 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:43.309043 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:43.309905 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:43.309925 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:43.311893 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:43.314220 (+ 2327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:43.314227 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:43.314252 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:43.314621 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:43.314971 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:43.314976 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:43.314978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:43.315140 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:43.315616 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:43.315625 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:43.315936 (+ 311us) client_negotiation.cc:769] Sending connection context
0619 02:29:43.316156 (+ 220us) client_negotiation.cc:240] Negotiation successful
0619 02:29:43.316573 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":161,"threads_started":1}
I20260619 02:29:43.319057 21181 heartbeater.cc:347] Connected to a master server at 127.17.96.254:45763
I20260619 02:29:43.319358 21181 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:43.319950 21181 heartbeater.cc:511] Master 127.17.96.254:45763 requested a full tablet report, sending...
I20260619 02:29:43.321264 20856 ts_manager.cc:194] Registered new tserver with Master: 208fee193dc6475b82bd435959b3a050 (127.17.96.194:42639)
I20260619 02:29:43.322299 20856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:47913
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
I20260619 02:29:43.332132 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45763
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:33893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:43.482478 21186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:43.482860 21186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:43.482967 21186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:43.487708 21186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:43.487862 21186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:43.487987 21186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:29:43.494234 21186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:33893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:43.495601 21186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:43.496835 21186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:43.508061 21191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:43.508066 21192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:43.508066 21194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:43.508347 21186 server_base.cc:1034] running on GCE node
I20260619 02:29:43.508940 21186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:43.509810 21186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:43.510991 21186 hybrid_clock.cc:648] HybridClock initialized: now 1781836183510951 us; error 58 us; skew 500 ppm
Jun 19 02:29:43 dist-test-slave-4qw6 krb5kdc[20811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836183, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:43.514355 21186 init.cc:377] Logged in from keytab as kudu/127.17.96.195@KRBTEST.COM (short username kudu)
I20260619 02:29:43.515517 21186 webserver.cc:460] Webserver started at http://127.17.96.195:37537/ using document root <none> and password file <none>
I20260619 02:29:43.516187 21186 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:43.516292 21186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:43.516557 21186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:43.518993 21186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "bd6c4051043f4672adfa46d9a9445bf2"
format_stamp: "Formatted at 2026-06-19 02:29:43 on dist-test-slave-4qw6"
server_key: "18b77897db52411ebb179a14f7599404"
server_key_iv: "66d636253dd95871396f995ca3211dd9"
server_key_version: "encryptionkey@0"
I20260619 02:29:43.519640 21186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "bd6c4051043f4672adfa46d9a9445bf2"
format_stamp: "Formatted at 2026-06-19 02:29:43 on dist-test-slave-4qw6"
server_key: "18b77897db52411ebb179a14f7599404"
server_key_iv: "66d636253dd95871396f995ca3211dd9"
server_key_version: "encryptionkey@0"
I20260619 02:29:43.524683 21186 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:29:43.528349 21201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:43.530040 21186 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:29:43.530210 21186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "bd6c4051043f4672adfa46d9a9445bf2"
format_stamp: "Formatted at 2026-06-19 02:29:43 on dist-test-slave-4qw6"
server_key: "18b77897db52411ebb179a14f7599404"
server_key_iv: "66d636253dd95871396f995ca3211dd9"
server_key_version: "encryptionkey@0"
I20260619 02:29:43.530355 21186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:43.564415 21186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:43.569265 21186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:43.569533 21186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:43.570295 21186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:43.571493 21186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:43.571569 21186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:43.571647 21186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:43.571696 21186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:43.582536 21186 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:41373
I20260619 02:29:43.582619 21314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:41373 every 8 connection(s)
I20260619 02:29:43.583657 21186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:29:43.591521 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 21186
I20260619 02:29:43.591710 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:29:43.592057 17795 external_mini_cluster.cc:1428] Setting key 329d52bdf1786b34913db03edd73be2e
Jun 19 02:29:43 dist-test-slave-4qw6 krb5kdc[20811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836183, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:43.600942 20895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:43.585869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:52495 (local address 127.17.96.254:45763)
0619 02:29:43.586084 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:43.586088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:43.587508 (+ 1420us) server_negotiation.cc:407] Connection header received
0619 02:29:43.589079 (+ 1571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:43.589083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:43.589147 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:43.589246 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:43.590776 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:43.591374 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:43.592855 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:43.593107 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:43.593236 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:43.596245 (+ 3009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:43.596249 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:43.596251 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:43.596279 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:43.598088 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:43.599044 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:43.599048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:43.599050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:43.599149 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:43.599580 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:43.599584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:43.599586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:43.599814 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:43.599940 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:43.600581 (+ 641us) server_negotiation.cc:299] Negotiation successful
0619 02:29:43.600746 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260619 02:29:43.602021 21317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:43.586575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45763 (local address 127.17.96.195:52495)
0619 02:29:43.587303 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:43.587361 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:43.588816 (+ 1455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:43.589475 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:43.589484 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:43.589944 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:43.590587 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:43.590604 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:43.591530 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:43.591534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:43.592692 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:43.592705 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:43.593294 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:43.593300 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:43.593508 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:43.594333 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:43.594355 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:43.596028 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:43.598264 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:43.598273 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:43.598299 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:43.598847 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:43.599265 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:43.599269 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:43.599271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:43.599454 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:43.599941 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:43.599952 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:43.600281 (+ 329us) client_negotiation.cc:769] Sending connection context
0619 02:29:43.600512 (+ 231us) client_negotiation.cc:240] Negotiation successful
0619 02:29:43.600925 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":196,"threads_started":1}
I20260619 02:29:43.603471 21315 heartbeater.cc:347] Connected to a master server at 127.17.96.254:45763
I20260619 02:29:43.603768 21315 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:43.604370 21315 heartbeater.cc:511] Master 127.17.96.254:45763 requested a full tablet report, sending...
I20260619 02:29:43.605599 20855 ts_manager.cc:194] Registered new tserver with Master: bd6c4051043f4672adfa46d9a9445bf2 (127.17.96.195:41373)
I20260619 02:29:43.606307 20855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:52495
I20260619 02:29:43.606554 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 02:29:43 dist-test-slave-4qw6 krb5kdc[20811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836183, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 19 02:29:43 dist-test-slave-4qw6 krb5kdc[20811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836183, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:43.637625 20895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:43.627250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45562 (local address 127.17.96.254:45763)
0619 02:29:43.627421 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:43.627424 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:43.627589 (+ 165us) server_negotiation.cc:407] Connection header received
0619 02:29:43.627750 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:43.627754 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:43.627810 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:43.628181 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:43.629006 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:43.629571 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:43.630591 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:43.630843 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:43.630975 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:43.633713 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:43.633717 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:43.633719 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:43.633828 (+ 109us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:43.635497 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:43.635984 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:43.635987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:43.635988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:43.636071 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:43.636382 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:43.636386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:43.636387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:43.636664 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:43.636794 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:43.637248 (+ 454us) server_negotiation.cc:299] Negotiation successful
0619 02:29:43.637440 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260619 02:29:43.638224 20855 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45562
I20260619 02:29:43.639271 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 20918
I20260619 02:29:43.645927 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 21052
I20260619 02:29:43.652385 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 21186
I20260619 02:29:43.658116 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 20824
2026-06-19T02:29:43Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2441 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:29:43 dist-test-slave-4qw6 krb5kdc[21327](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:29:43 dist-test-slave-4qw6 krb5kdc[21327](info): set up 2 sockets
Jun 19 02:29:43 dist-test-slave-4qw6 krb5kdc[21327](info): commencing operation
krb5kdc: starting...
W20260619 02:29:44.790284 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:29:44 dist-test-slave-4qw6 krb5kdc[21327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:29:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:29:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:29:44.933637 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:35093
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:34667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:35093
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:45.061717 21340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:45.062093 21340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:45.062142 21340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:45.066911 21340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:29:45.067003 21340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:45.067026 21340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:45.067045 21340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:29:45.067063 21340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:29:45.073294 21340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:34667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:35093
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:35093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:45.074630 21340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:45.075798 21340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:45.086270 21346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:45.086270 21345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:45.086302 21348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:45.086807 21340 server_base.cc:1034] running on GCE node
I20260619 02:29:45.087517 21340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:45.088881 21340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:45.090164 21340 hybrid_clock.cc:648] HybridClock initialized: now 1781836185090140 us; error 49 us; skew 500 ppm
Jun 19 02:29:45 dist-test-slave-4qw6 krb5kdc[21327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836185, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:45.093869 21340 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:29:45.095075 21340 webserver.cc:460] Webserver started at http://127.17.96.254:35103/ using document root <none> and password file <none>
I20260619 02:29:45.095780 21340 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:45.095894 21340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:45.096174 21340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:45.098913 21340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "0f40cc7afe5a4656af705ea166c7fffd"
format_stamp: "Formatted at 2026-06-19 02:29:45 on dist-test-slave-4qw6"
server_key: "71dd8125f2dd8a08502d819cc01526ce"
server_key_iv: "60443f610ff147fa8acbb808b923e443"
server_key_version: "encryptionkey@0"
I20260619 02:29:45.099603 21340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "0f40cc7afe5a4656af705ea166c7fffd"
format_stamp: "Formatted at 2026-06-19 02:29:45 on dist-test-slave-4qw6"
server_key: "71dd8125f2dd8a08502d819cc01526ce"
server_key_iv: "60443f610ff147fa8acbb808b923e443"
server_key_version: "encryptionkey@0"
I20260619 02:29:45.104687 21340 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:29:45.107755 21355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:45.109238 21340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:29:45.109436 21340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "0f40cc7afe5a4656af705ea166c7fffd"
format_stamp: "Formatted at 2026-06-19 02:29:45 on dist-test-slave-4qw6"
server_key: "71dd8125f2dd8a08502d819cc01526ce"
server_key_iv: "60443f610ff147fa8acbb808b923e443"
server_key_version: "encryptionkey@0"
I20260619 02:29:45.109589 21340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:45.143941 21340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:45.148243 21340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:45.148510 21340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:45.157531 21340 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:35093
I20260619 02:29:45.157567 21407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:35093 every 8 connection(s)
I20260619 02:29:45.158752 21340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:29:45.161908 21408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:45.162537 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 21340
I20260619 02:29:45.162691 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:29:45.163014 17795 external_mini_cluster.cc:1428] Setting key 5bf7ab0fd8f7a0227a07abb6ea3f0ce4
I20260619 02:29:45.169471 21408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f40cc7afe5a4656af705ea166c7fffd: Bootstrap starting.
I20260619 02:29:45.172880 21408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f40cc7afe5a4656af705ea166c7fffd: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:45.174026 21408 log.cc:824] T 00000000000000000000000000000000 P 0f40cc7afe5a4656af705ea166c7fffd: Log is configured to *not* fsync() on all Append() calls
Jun 19 02:29:45 dist-test-slave-4qw6 krb5kdc[21327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:45.177364 21408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f40cc7afe5a4656af705ea166c7fffd: No bootstrap required, opened a new log
I20260619 02:29:45.182106 21411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:45.164758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52502 (local address 127.17.96.254:35093)
0619 02:29:45.165314 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:45.165334 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:45.165378 (+ 44us) server_negotiation.cc:407] Connection header received
0619 02:29:45.166222 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:45.166244 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:45.166661 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:45.167033 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:45.168143 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:45.169302 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:45.170661 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:45.171231 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:45.171390 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:45.175425 (+ 4035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:45.175435 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:45.175459 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:45.175494 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:45.178311 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:45.178912 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:45.178919 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:45.178926 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:45.179083 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:45.179372 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:45.179376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:45.179378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:45.180000 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:45.180248 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:45.180574 (+ 326us) server_negotiation.cc:299] Negotiation successful
0619 02:29:45.180924 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":200,"threads_started":1}
I20260619 02:29:45.182823 21408 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f40cc7afe5a4656af705ea166c7fffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f40cc7afe5a4656af705ea166c7fffd" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35093 } }
I20260619 02:29:45.183132 21408 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f40cc7afe5a4656af705ea166c7fffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:45.183183 21408 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f40cc7afe5a4656af705ea166c7fffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f40cc7afe5a4656af705ea166c7fffd, State: Initialized, Role: FOLLOWER
I20260619 02:29:45.183683 21408 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f40cc7afe5a4656af705ea166c7fffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f40cc7afe5a4656af705ea166c7fffd" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35093 } }
I20260619 02:29:45.183854 21408 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f40cc7afe5a4656af705ea166c7fffd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:29:45.183924 21408 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f40cc7afe5a4656af705ea166c7fffd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:29:45.184026 21408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f40cc7afe5a4656af705ea166c7fffd [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:45.185299 21408 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f40cc7afe5a4656af705ea166c7fffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f40cc7afe5a4656af705ea166c7fffd" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35093 } }
I20260619 02:29:45.185696 21408 leader_election.cc:302] T 00000000000000000000000000000000 P 0f40cc7afe5a4656af705ea166c7fffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f40cc7afe5a4656af705ea166c7fffd; no voters:
I20260619 02:29:45.186122 21408 leader_election.cc:288] T 00000000000000000000000000000000 P 0f40cc7afe5a4656af705ea166c7fffd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:29:45.186221 21413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f40cc7afe5a4656af705ea166c7fffd [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:45.186405 21413 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f40cc7afe5a4656af705ea166c7fffd [term 1 LEADER]: Becoming Leader. State: Replica: 0f40cc7afe5a4656af705ea166c7fffd, State: Running, Role: LEADER
I20260619 02:29:45.186770 21413 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f40cc7afe5a4656af705ea166c7fffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f40cc7afe5a4656af705ea166c7fffd" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35093 } }
I20260619 02:29:45.187639 21408 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f40cc7afe5a4656af705ea166c7fffd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:29:45.188789 21414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f40cc7afe5a4656af705ea166c7fffd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f40cc7afe5a4656af705ea166c7fffd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f40cc7afe5a4656af705ea166c7fffd" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35093 } } }
I20260619 02:29:45.188846 21415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f40cc7afe5a4656af705ea166c7fffd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f40cc7afe5a4656af705ea166c7fffd. Latest consensus state: current_term: 1 leader_uuid: "0f40cc7afe5a4656af705ea166c7fffd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f40cc7afe5a4656af705ea166c7fffd" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35093 } } }
I20260619 02:29:45.189074 21415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f40cc7afe5a4656af705ea166c7fffd [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:45.189116 21414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f40cc7afe5a4656af705ea166c7fffd [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:45.189713 21422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:29:45.193555 21422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:29:45.201299 21422 catalog_manager.cc:1347] Generated new cluster ID: 9d86577a51564b0299d85c6d3ce4a317
I20260619 02:29:45.201434 21422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:29:45.215739 21422 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:29:45.216881 21422 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:29:45.223304 21422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f40cc7afe5a4656af705ea166c7fffd: Generated new TSK 0
I20260619 02:29:45.224205 21422 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.96.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
I20260619 02:29:45.262879 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:35093
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:34667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:45.396314 21434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:45.396651 21434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:45.396699 21434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:45.401116 21434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:45.401193 21434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:45.401260 21434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:29:45.407284 21434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:34667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:35093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:45.408524 21434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:45.409649 21434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:45.420981 21440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:45.421046 21442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:45.421008 21439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:45.421993 21434 server_base.cc:1034] running on GCE node
I20260619 02:29:45.422544 21434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:45.423326 21434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:45.424542 21434 hybrid_clock.cc:648] HybridClock initialized: now 1781836185424524 us; error 35 us; skew 500 ppm
Jun 19 02:29:45 dist-test-slave-4qw6 krb5kdc[21327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836185, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:45.427969 21434 init.cc:377] Logged in from keytab as kudu/127.17.96.193@KRBTEST.COM (short username kudu)
I20260619 02:29:45.429036 21434 webserver.cc:460] Webserver started at http://127.17.96.193:43571/ using document root <none> and password file <none>
I20260619 02:29:45.429729 21434 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:45.429885 21434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:45.430179 21434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:45.432651 21434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "5b26b5b243d54d66bcef9b3fbc9799e9"
format_stamp: "Formatted at 2026-06-19 02:29:45 on dist-test-slave-4qw6"
server_key: "d2c2c3025586cf0408adc51f84e01195"
server_key_iv: "8ff4b12c86538bf139308a43d5affd22"
server_key_version: "encryptionkey@0"
I20260619 02:29:45.433337 21434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "5b26b5b243d54d66bcef9b3fbc9799e9"
format_stamp: "Formatted at 2026-06-19 02:29:45 on dist-test-slave-4qw6"
server_key: "d2c2c3025586cf0408adc51f84e01195"
server_key_iv: "8ff4b12c86538bf139308a43d5affd22"
server_key_version: "encryptionkey@0"
I20260619 02:29:45.439201 21434 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260619 02:29:45.443305 21449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:45.445292 21434 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260619 02:29:45.445562 21434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "5b26b5b243d54d66bcef9b3fbc9799e9"
format_stamp: "Formatted at 2026-06-19 02:29:45 on dist-test-slave-4qw6"
server_key: "d2c2c3025586cf0408adc51f84e01195"
server_key_iv: "8ff4b12c86538bf139308a43d5affd22"
server_key_version: "encryptionkey@0"
I20260619 02:29:45.445807 21434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:45.472257 21434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:45.476701 21434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:45.477041 21434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:45.478138 21434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:45.479499 21434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:45.479643 21434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:45.479730 21434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:45.479780 21434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:45.490653 21434 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:39739
I20260619 02:29:45.490715 21562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:39739 every 8 connection(s)
I20260619 02:29:45.491883 21434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:29:45.501924 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 21434
I20260619 02:29:45.502068 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:29:45.502430 17795 external_mini_cluster.cc:1428] Setting key f8e8e9287face52e2287ef35aeca3bbf
Jun 19 02:29:45 dist-test-slave-4qw6 krb5kdc[21327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836185, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:45.510313 21411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:45.494230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:47647 (local address 127.17.96.254:35093)
0619 02:29:45.494386 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:45.494391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:45.496297 (+ 1906us) server_negotiation.cc:407] Connection header received
0619 02:29:45.497346 (+ 1049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:45.497349 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:45.497409 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:45.497509 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:45.499034 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:45.499667 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:45.501026 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:45.501311 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:45.501556 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:45.505249 (+ 3693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:45.505256 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:45.505263 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:45.505309 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:45.507245 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:45.508211 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:45.508218 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:45.508223 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:45.508329 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:45.508753 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:45.508760 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:45.508765 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:45.509077 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:45.509199 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:45.509855 (+ 656us) server_negotiation.cc:299] Negotiation successful
0619 02:29:45.510042 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 02:29:45.511312 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:45.495064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:35093 (local address 127.17.96.193:47647)
0619 02:29:45.496055 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:45.496111 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:45.497045 (+ 934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:45.497740 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:45.497815 (+ 75us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:45.498241 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:45.498801 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:45.498816 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:45.499846 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:45.499848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:45.500827 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:45.500838 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:45.501609 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:45.501612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:45.501861 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:45.502634 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:45.502652 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:45.504994 (+ 2342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:45.507446 (+ 2452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:45.507461 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:45.507487 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:45.508065 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:45.508440 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:45.508442 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:45.508444 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:45.508633 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:45.509200 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:45.509206 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:45.509518 (+ 312us) client_negotiation.cc:769] Sending connection context
0619 02:29:45.509724 (+ 206us) client_negotiation.cc:240] Negotiation successful
0619 02:29:45.510194 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"thread_start_us":197,"threads_started":1}
I20260619 02:29:45.512921 21563 heartbeater.cc:347] Connected to a master server at 127.17.96.254:35093
I20260619 02:29:45.513238 21563 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:45.513950 21563 heartbeater.cc:511] Master 127.17.96.254:35093 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.96.194@KRBTEST.COM; defaulting to no policy
I20260619 02:29:45.515959 21372 ts_manager.cc:194] Registered new tserver with Master: 5b26b5b243d54d66bcef9b3fbc9799e9 (127.17.96.193:39739)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.194@KRBTEST.COM" created.
I20260619 02:29:45.517351 21372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:47647
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
I20260619 02:29:45.537315 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:35093
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:34667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:45.664444 21568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:45.664798 21568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:45.664849 21568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:45.669857 21568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:45.669955 21568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:45.670023 21568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:29:45.676433 21568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:34667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:35093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:45.677707 21568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:45.678929 21568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:45.690554 21574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:45.690531 21573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:45.690532 21576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:45.691360 21568 server_base.cc:1034] running on GCE node
I20260619 02:29:45.691943 21568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:45.692840 21568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:45.694084 21568 hybrid_clock.cc:648] HybridClock initialized: now 1781836185694060 us; error 45 us; skew 500 ppm
Jun 19 02:29:45 dist-test-slave-4qw6 krb5kdc[21327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836185, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:45.697373 21568 init.cc:377] Logged in from keytab as kudu/127.17.96.194@KRBTEST.COM (short username kudu)
I20260619 02:29:45.698642 21568 webserver.cc:460] Webserver started at http://127.17.96.194:36463/ using document root <none> and password file <none>
I20260619 02:29:45.699271 21568 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:45.699350 21568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:45.699548 21568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:45.701984 21568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "38e439798da942eeb4acc310ddbf6531"
format_stamp: "Formatted at 2026-06-19 02:29:45 on dist-test-slave-4qw6"
server_key: "e0f7296d66b70b00d1856cd6fe643c96"
server_key_iv: "9965008f52d793aabd152e6a1229af19"
server_key_version: "encryptionkey@0"
I20260619 02:29:45.702636 21568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "38e439798da942eeb4acc310ddbf6531"
format_stamp: "Formatted at 2026-06-19 02:29:45 on dist-test-slave-4qw6"
server_key: "e0f7296d66b70b00d1856cd6fe643c96"
server_key_iv: "9965008f52d793aabd152e6a1229af19"
server_key_version: "encryptionkey@0"
I20260619 02:29:45.707517 21568 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260619 02:29:45.710902 21583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:45.712493 21568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 02:29:45.712689 21568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "38e439798da942eeb4acc310ddbf6531"
format_stamp: "Formatted at 2026-06-19 02:29:45 on dist-test-slave-4qw6"
server_key: "e0f7296d66b70b00d1856cd6fe643c96"
server_key_iv: "9965008f52d793aabd152e6a1229af19"
server_key_version: "encryptionkey@0"
I20260619 02:29:45.712843 21568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:45.725942 21568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:45.730422 21568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:45.730733 21568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:45.731534 21568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:45.732695 21568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:45.732766 21568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:45.732820 21568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:45.732988 21568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:45.746690 21568 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:43703
I20260619 02:29:45.746709 21696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:43703 every 8 connection(s)
I20260619 02:29:45.748080 21568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:29:45.757264 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 21568
I20260619 02:29:45.757417 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:29:45.757791 17795 external_mini_cluster.cc:1428] Setting key cadd03474c9d212afbaf46fcd44e16bc
Jun 19 02:29:45 dist-test-slave-4qw6 krb5kdc[21327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836185, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:45.767724 21411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:45.750683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:42693 (local address 127.17.96.254:35093)
0619 02:29:45.750851 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:45.750855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:45.752338 (+ 1483us) server_negotiation.cc:407] Connection header received
0619 02:29:45.754181 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:45.754185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:45.754254 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:45.754383 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:45.755908 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:45.756512 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:45.758440 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:45.758748 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:45.758885 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:45.763247 (+ 4362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:45.763252 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:45.763255 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:45.763291 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:45.765122 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:45.765840 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:45.765844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:45.765846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:45.765943 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:45.766369 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:45.766373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:45.766375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:45.766608 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:45.766743 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:45.767378 (+ 635us) server_negotiation.cc:299] Negotiation successful
0619 02:29:45.767534 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:29:45.769114 21699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:45.751218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:35093 (local address 127.17.96.194:42693)
0619 02:29:45.751990 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:45.752062 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:45.753891 (+ 1829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:45.754609 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:45.754618 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:45.755083 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:45.755701 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:45.755718 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:45.756672 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:45.756676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:45.758263 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:45.758284 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:45.759147 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:45.759153 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:45.759514 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:45.760553 (+ 1039us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:45.760582 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:45.762958 (+ 2376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:45.765290 (+ 2332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:45.765299 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:45.765331 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:45.765681 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:45.766059 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:45.766063 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:45.766065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:45.766249 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:45.766755 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:45.766765 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:45.767091 (+ 326us) client_negotiation.cc:769] Sending connection context
0619 02:29:45.767316 (+ 225us) client_negotiation.cc:240] Negotiation successful
0619 02:29:45.767751 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":162,"threads_started":1}
WARNING: no policy specified for kudu/127.17.96.195@KRBTEST.COM; defaulting to no policy
I20260619 02:29:45.770910 21697 heartbeater.cc:347] Connected to a master server at 127.17.96.254:35093
I20260619 02:29:45.771270 21697 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:45.772217 21697 heartbeater.cc:511] Master 127.17.96.254:35093 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.195@KRBTEST.COM" created.
I20260619 02:29:45.774210 21372 ts_manager.cc:194] Registered new tserver with Master: 38e439798da942eeb4acc310ddbf6531 (127.17.96.194:43703)
I20260619 02:29:45.775089 21372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:42693
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
I20260619 02:29:45.794970 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:35093
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:34667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:45.923740 21702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:45.924074 21702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:45.924127 21702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:45.928580 21702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:45.928687 21702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:45.928759 21702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:29:45.936110 21702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:34667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:35093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:45.937422 21702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:45.938629 21702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:29:45.950088 21702 server_base.cc:1034] running on GCE node
W20260619 02:29:45.950031 21707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:45.950034 21708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:45.950031 21710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:45.950927 21702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:45.951859 21702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:45.953056 21702 hybrid_clock.cc:648] HybridClock initialized: now 1781836185953038 us; error 36 us; skew 500 ppm
Jun 19 02:29:45 dist-test-slave-4qw6 krb5kdc[21327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836185, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:45.956635 21702 init.cc:377] Logged in from keytab as kudu/127.17.96.195@KRBTEST.COM (short username kudu)
I20260619 02:29:45.958166 21702 webserver.cc:460] Webserver started at http://127.17.96.195:41891/ using document root <none> and password file <none>
I20260619 02:29:45.959160 21702 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:45.959312 21702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:45.959654 21702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:45.962275 21702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "2ef4f9691ee748dc8d2aa8edfa2571cb"
format_stamp: "Formatted at 2026-06-19 02:29:45 on dist-test-slave-4qw6"
server_key: "5305fafa7cc4b5f2fe659f221945850f"
server_key_iv: "adfe360a57d56d826f5c6a6f31039668"
server_key_version: "encryptionkey@0"
I20260619 02:29:45.963038 21702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "2ef4f9691ee748dc8d2aa8edfa2571cb"
format_stamp: "Formatted at 2026-06-19 02:29:45 on dist-test-slave-4qw6"
server_key: "5305fafa7cc4b5f2fe659f221945850f"
server_key_iv: "adfe360a57d56d826f5c6a6f31039668"
server_key_version: "encryptionkey@0"
I20260619 02:29:45.967905 21702 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:29:45.971530 21717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:45.973285 21702 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260619 02:29:45.973515 21702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "2ef4f9691ee748dc8d2aa8edfa2571cb"
format_stamp: "Formatted at 2026-06-19 02:29:45 on dist-test-slave-4qw6"
server_key: "5305fafa7cc4b5f2fe659f221945850f"
server_key_iv: "adfe360a57d56d826f5c6a6f31039668"
server_key_version: "encryptionkey@0"
I20260619 02:29:45.973677 21702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:46.005975 21702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:46.009522 21702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:46.009840 21702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:46.010618 21702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:46.011658 21702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:46.011749 21702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:46.011837 21702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:46.011888 21702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:46.023226 21702 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:46407
I20260619 02:29:46.023316 21830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:46407 every 8 connection(s)
I20260619 02:29:46.024435 21702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:29:46.034880 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 21702
I20260619 02:29:46.035094 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:29:46.035650 17795 external_mini_cluster.cc:1428] Setting key 792fd0d056ee9fd8d44fb508336faf25
Jun 19 02:29:46 dist-test-slave-4qw6 krb5kdc[21327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836185, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:46.041131 21411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:46.026736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:54489 (local address 127.17.96.254:35093)
0619 02:29:46.026912 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:46.026917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:46.027940 (+ 1023us) server_negotiation.cc:407] Connection header received
0619 02:29:46.029312 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:46.029315 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:46.029370 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:46.029455 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:46.030980 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:46.031558 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:46.032918 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:46.033167 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:46.033271 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:46.036832 (+ 3561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:46.036838 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:46.036846 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:46.036878 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:46.038869 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:46.039545 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:46.039548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:46.039549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:46.039636 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:46.040007 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:46.040009 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:46.040011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:46.040218 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:46.040301 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:46.040869 (+ 568us) server_negotiation.cc:299] Negotiation successful
0619 02:29:46.040991 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 02:29:46.042264 21833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:46.027211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:35093 (local address 127.17.96.195:54489)
0619 02:29:46.027733 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:46.027784 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:46.029069 (+ 1285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:46.029660 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:46.029667 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:46.030186 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:46.030775 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:46.030792 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:46.031741 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:46.031745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:46.032742 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:46.032753 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:46.033332 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:46.033335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:46.033500 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:46.034277 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:46.034296 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:46.036620 (+ 2324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:46.039030 (+ 2410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:46.039039 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:46.039069 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:46.039431 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:46.039732 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:46.039735 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:46.039736 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:46.039902 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:46.040317 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:46.040323 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:46.040624 (+ 301us) client_negotiation.cc:769] Sending connection context
0619 02:29:46.040821 (+ 197us) client_negotiation.cc:240] Negotiation successful
0619 02:29:46.041176 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":118,"threads_started":1}
I20260619 02:29:46.043583 21831 heartbeater.cc:347] Connected to a master server at 127.17.96.254:35093
I20260619 02:29:46.043836 21831 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:46.044365 21831 heartbeater.cc:511] Master 127.17.96.254:35093 requested a full tablet report, sending...
I20260619 02:29:46.045542 21372 ts_manager.cc:194] Registered new tserver with Master: 2ef4f9691ee748dc8d2aa8edfa2571cb (127.17.96.195:46407)
I20260619 02:29:46.046264 21372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:54489
I20260619 02:29:46.051566 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 02:29:46 dist-test-slave-4qw6 krb5kdc[21327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:29:46 dist-test-slave-4qw6 krb5kdc[21327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.193@KRBTEST.COM
I20260619 02:29:46.083560 21838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:46.072530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37942 (local address 127.17.96.193:39739)
0619 02:29:46.072882 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:46.072890 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:46.072916 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:29:46.072975 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:46.072981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:46.073241 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:46.073393 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:46.074353 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:46.075234 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:46.076186 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:46.076598 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:46.076688 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:46.079635 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:46.079644 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:46.079662 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:46.079693 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:46.081420 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:46.081997 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:46.082002 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:46.082003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:46.082088 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:46.082393 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:46.082396 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:46.082398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:46.082674 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:46.082935 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:46.083175 (+ 240us) server_negotiation.cc:299] Negotiation successful
0619 02:29:46.083357 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":124,"threads_started":1}
Jun 19 02:29:46 dist-test-slave-4qw6 krb5kdc[21327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:46.099325 21411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:46.086884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52514 (local address 127.17.96.254:35093)
0619 02:29:46.087153 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:46.087160 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:46.087304 (+ 144us) server_negotiation.cc:407] Connection header received
0619 02:29:46.087571 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:46.087574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:46.087629 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:46.087735 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:46.088593 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:46.089474 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:46.090708 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:46.090974 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:46.091110 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:46.094437 (+ 3327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:46.094442 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:46.094444 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:46.094490 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:46.096135 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:46.097239 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:46.097244 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:46.097246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:46.097479 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:46.098106 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:46.098111 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:46.098112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:46.098408 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:46.098561 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:46.098972 (+ 411us) server_negotiation.cc:299] Negotiation successful
0619 02:29:46.099105 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
W20260619 02:29:46.099865 21372 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52514
I20260619 02:29:46.100641 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 21434
I20260619 02:29:46.107249 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 21568
I20260619 02:29:46.113422 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 21702
I20260619 02:29:46.121371 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 21340
2026-06-19T02:29:46Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2464 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:29:46 dist-test-slave-4qw6 krb5kdc[21843](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:29:46 dist-test-slave-4qw6 krb5kdc[21843](info): set up 2 sockets
Jun 19 02:29:46 dist-test-slave-4qw6 krb5kdc[21843](info): commencing operation
krb5kdc: starting...
W20260619 02:29:47.250382 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:29:47 dist-test-slave-4qw6 krb5kdc[21843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:29:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:29:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:29:47.393821 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:45725
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:40549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:45725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:47.521418 21856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:47.521842 21856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:47.521914 21856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:47.526535 21856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:29:47.526614 21856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:47.526635 21856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:47.526654 21856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:29:47.526701 21856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:29:47.533059 21856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:40549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:45725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:45725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:47.534524 21856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:47.535749 21856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:47.545853 21861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:47.545853 21862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:47.545853 21864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:47.545948 21856 server_base.cc:1034] running on GCE node
I20260619 02:29:47.546793 21856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:47.548065 21856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:47.549278 21856 hybrid_clock.cc:648] HybridClock initialized: now 1781836187549257 us; error 40 us; skew 500 ppm
Jun 19 02:29:47 dist-test-slave-4qw6 krb5kdc[21843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836187, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:47.552525 21856 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:29:47.553447 21856 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:47.553521 21856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:47.553871 21856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:47.556340 21856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "f99b9a7fc89a4ffd8b6ee01eee15b9a2"
format_stamp: "Formatted at 2026-06-19 02:29:47 on dist-test-slave-4qw6"
server_key: "eedfda9353253c5d0cff37ea7f963f15"
server_key_iv: "ffc007949ac81b3a86e83d1e5f606ee3"
server_key_version: "encryptionkey@0"
I20260619 02:29:47.557021 21856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "f99b9a7fc89a4ffd8b6ee01eee15b9a2"
format_stamp: "Formatted at 2026-06-19 02:29:47 on dist-test-slave-4qw6"
server_key: "eedfda9353253c5d0cff37ea7f963f15"
server_key_iv: "ffc007949ac81b3a86e83d1e5f606ee3"
server_key_version: "encryptionkey@0"
I20260619 02:29:47.561689 21856 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 02:29:47.564776 21870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:47.566346 21856 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:29:47.566517 21856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "f99b9a7fc89a4ffd8b6ee01eee15b9a2"
format_stamp: "Formatted at 2026-06-19 02:29:47 on dist-test-slave-4qw6"
server_key: "eedfda9353253c5d0cff37ea7f963f15"
server_key_iv: "ffc007949ac81b3a86e83d1e5f606ee3"
server_key_version: "encryptionkey@0"
I20260619 02:29:47.566658 21856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:47.583772 21856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:47.587292 21856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:47.587551 21856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:47.595572 21856 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:45725
I20260619 02:29:47.595890 21922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:45725 every 8 connection(s)
I20260619 02:29:47.596411 21856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:29:47.599638 21923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:47.602487 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 21856
I20260619 02:29:47.602638 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:29:47.602960 17795 external_mini_cluster.cc:1428] Setting key c4f5f0b9790f167726d51dc055bc153f
I20260619 02:29:47.606945 21923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2: Bootstrap starting.
I20260619 02:29:47.609912 21923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:47.610900 21923 log.cc:824] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2: Log is configured to *not* fsync() on all Append() calls
Jun 19 02:29:47 dist-test-slave-4qw6 krb5kdc[21843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:47.613624 21923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2: No bootstrap required, opened a new log
I20260619 02:29:47.617691 21923 raft_consensus.cc:348] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f99b9a7fc89a4ffd8b6ee01eee15b9a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45725 } }
I20260619 02:29:47.618090 21923 raft_consensus.cc:374] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:47.618155 21923 raft_consensus.cc:729] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99b9a7fc89a4ffd8b6ee01eee15b9a2, State: Initialized, Role: FOLLOWER
I20260619 02:29:47.618742 21923 consensus_queue.cc:260] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f99b9a7fc89a4ffd8b6ee01eee15b9a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45725 } }
I20260619 02:29:47.618945 21923 raft_consensus.cc:388] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:29:47.619019 21923 raft_consensus.cc:482] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:29:47.619122 21923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:47.619947 21926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:47.604601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57794 (local address 127.17.96.254:45725)
0619 02:29:47.605155 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:47.605179 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:47.605236 (+ 57us) server_negotiation.cc:407] Connection header received
0619 02:29:47.606152 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:47.606173 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:47.606543 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:47.606853 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:47.607848 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:47.608892 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:47.610171 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:47.610713 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:47.610855 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:47.614091 (+ 3236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:47.614098 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:47.614114 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:47.614147 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:47.616235 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:47.616814 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:47.616820 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:47.616827 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:47.616970 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:47.617307 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:47.617312 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:47.617314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:47.617887 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:47.618147 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:47.618468 (+ 321us) server_negotiation.cc:299] Negotiation successful
0619 02:29:47.618822 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":143,"threads_started":1}
I20260619 02:29:47.620496 21923 raft_consensus.cc:504] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f99b9a7fc89a4ffd8b6ee01eee15b9a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45725 } }
I20260619 02:29:47.620888 21923 leader_election.cc:302] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f99b9a7fc89a4ffd8b6ee01eee15b9a2; no voters:
I20260619 02:29:47.621279 21923 leader_election.cc:288] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:29:47.621400 21928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:47.621656 21928 raft_consensus.cc:686] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2 [term 1 LEADER]: Becoming Leader. State: Replica: f99b9a7fc89a4ffd8b6ee01eee15b9a2, State: Running, Role: LEADER
I20260619 02:29:47.622076 21928 consensus_queue.cc:237] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f99b9a7fc89a4ffd8b6ee01eee15b9a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45725 } }
I20260619 02:29:47.622709 21923 sys_catalog.cc:564] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:29:47.624154 21929 sys_catalog.cc:455] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f99b9a7fc89a4ffd8b6ee01eee15b9a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f99b9a7fc89a4ffd8b6ee01eee15b9a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45725 } } }
I20260619 02:29:47.624199 21930 sys_catalog.cc:455] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f99b9a7fc89a4ffd8b6ee01eee15b9a2. Latest consensus state: current_term: 1 leader_uuid: "f99b9a7fc89a4ffd8b6ee01eee15b9a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f99b9a7fc89a4ffd8b6ee01eee15b9a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45725 } } }
I20260619 02:29:47.624310 21929 sys_catalog.cc:458] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:47.624316 21930 sys_catalog.cc:458] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:47.624734 21937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:29:47.628283 21937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:29:47.634972 21937 catalog_manager.cc:1347] Generated new cluster ID: 06868f8b02d347bbbcabeb6e2b894f09
I20260619 02:29:47.635068 21937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:29:47.646106 21937 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:29:47.647305 21937 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:29:47.655973 21937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2: Generated new TSK 0
I20260619 02:29:47.657166 21937 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.96.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
I20260619 02:29:47.702040 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45725
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:40549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:47.859935 21949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:47.860334 21949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:47.860467 21949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:47.865145 21949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:47.865283 21949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:47.865410 21949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:29:47.872207 21949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:40549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:47.873608 21949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:47.874890 21949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:47.886260 21957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:47.886260 21955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:47.886353 21954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:47.886945 21949 server_base.cc:1034] running on GCE node
I20260619 02:29:47.887552 21949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:47.888378 21949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:47.889604 21949 hybrid_clock.cc:648] HybridClock initialized: now 1781836187889580 us; error 46 us; skew 500 ppm
Jun 19 02:29:47 dist-test-slave-4qw6 krb5kdc[21843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836187, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:47.896282 21949 init.cc:377] Logged in from keytab as kudu/127.17.96.193@KRBTEST.COM (short username kudu)
I20260619 02:29:47.897275 21949 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:47.897400 21949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:47.897686 21949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:47.900089 21949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "26a2b0e880394418b0b60071e082af33"
format_stamp: "Formatted at 2026-06-19 02:29:47 on dist-test-slave-4qw6"
server_key: "1213d6e82a30fec1c19a060da1346a31"
server_key_iv: "2d1d46040fc5c312790209ebc424c3dd"
server_key_version: "encryptionkey@0"
I20260619 02:29:47.900743 21949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "26a2b0e880394418b0b60071e082af33"
format_stamp: "Formatted at 2026-06-19 02:29:47 on dist-test-slave-4qw6"
server_key: "1213d6e82a30fec1c19a060da1346a31"
server_key_iv: "2d1d46040fc5c312790209ebc424c3dd"
server_key_version: "encryptionkey@0"
I20260619 02:29:47.905092 21949 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260619 02:29:47.908125 21963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:47.909735 21949 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:29:47.909979 21949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "26a2b0e880394418b0b60071e082af33"
format_stamp: "Formatted at 2026-06-19 02:29:47 on dist-test-slave-4qw6"
server_key: "1213d6e82a30fec1c19a060da1346a31"
server_key_iv: "2d1d46040fc5c312790209ebc424c3dd"
server_key_version: "encryptionkey@0"
I20260619 02:29:47.910143 21949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:47.927569 21949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:47.931885 21949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:47.932227 21949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:47.933058 21949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:47.934391 21949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:47.934538 21949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:47.934628 21949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:47.934675 21949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:47.945037 21949 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:32973
I20260619 02:29:47.945079 22076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:32973 every 8 connection(s)
I20260619 02:29:47.945791 21949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:29:47.952716 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 21949
I20260619 02:29:47.952843 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:29:47.953155 17795 external_mini_cluster.cc:1428] Setting key 3839fcc2001ad4ebebb02c278b1e401b
Jun 19 02:29:47 dist-test-slave-4qw6 krb5kdc[21843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836187, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:47.963053 21926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:47.948031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:37001 (local address 127.17.96.254:45725)
0619 02:29:47.948264 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:47.948272 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:47.949995 (+ 1723us) server_negotiation.cc:407] Connection header received
0619 02:29:47.950957 (+ 962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:47.950960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:47.951014 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:47.951104 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:47.952540 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:47.953223 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:47.954881 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:47.955246 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:47.955391 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:47.958862 (+ 3471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:47.958866 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:47.958868 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:47.958896 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:47.960541 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:47.961177 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:47.961186 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:47.961188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:47.961273 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:47.961671 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:47.961675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:47.961677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:47.961977 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:47.962098 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:47.962704 (+ 606us) server_negotiation.cc:299] Negotiation successful
0619 02:29:47.962853 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
WARNING: no policy specified for kudu/127.17.96.194@KRBTEST.COM; defaulting to no policy
I20260619 02:29:47.964051 22079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:47.948687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45725 (local address 127.17.96.193:37001)
0619 02:29:47.949728 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:47.949854 (+ 126us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:47.950737 (+ 883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:47.951294 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:47.951302 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:47.951730 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:47.952311 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:47.952327 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:47.953382 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:47.953386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:47.954738 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:47.954755 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:47.955495 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:47.955499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:47.955687 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:47.956438 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:47.956458 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:47.958617 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:47.960701 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:47.960710 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:47.960731 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:47.961053 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:47.961378 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:47.961382 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:47.961384 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:47.961557 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:47.962098 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:47.962107 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:47.962420 (+ 313us) client_negotiation.cc:769] Sending connection context
0619 02:29:47.962645 (+ 225us) client_negotiation.cc:240] Negotiation successful
0619 02:29:47.963036 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":178,"threads_started":1}
I20260619 02:29:47.965652 22077 heartbeater.cc:347] Connected to a master server at 127.17.96.254:45725
I20260619 02:29:47.965986 22077 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.194@KRBTEST.COM" created.
I20260619 02:29:47.966567 22077 heartbeater.cc:511] Master 127.17.96.254:45725 requested a full tablet report, sending...
I20260619 02:29:47.968549 21887 ts_manager.cc:194] Registered new tserver with Master: 26a2b0e880394418b0b60071e082af33 (127.17.96.193:32973)
I20260619 02:29:47.970386 21887 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:37001
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
I20260619 02:29:47.985396 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45725
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:40549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:48.117051 22082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:48.117388 22082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:48.117604 22082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:48.122865 22082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:48.122946 22082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:48.123070 22082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:29:48.131418 22082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:40549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:48.132815 22082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:48.134058 22082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:48.144835 22088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:48.144897 22090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:48.144887 22087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:48.145376 22082 server_base.cc:1034] running on GCE node
I20260619 02:29:48.145998 22082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:48.146831 22082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:48.148051 22082 hybrid_clock.cc:648] HybridClock initialized: now 1781836188148031 us; error 38 us; skew 500 ppm
Jun 19 02:29:48 dist-test-slave-4qw6 krb5kdc[21843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836188, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:48.151561 22082 init.cc:377] Logged in from keytab as kudu/127.17.96.194@KRBTEST.COM (short username kudu)
I20260619 02:29:48.152585 22082 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:48.152698 22082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:48.153038 22082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:48.155510 22082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "67a6e0a35fed4d5cacf97b11531c6e9a"
format_stamp: "Formatted at 2026-06-19 02:29:48 on dist-test-slave-4qw6"
server_key: "d6eaa605904af4954744d96caab322cb"
server_key_iv: "02bd0bb560656ef0ac69a5dbda0cb8d7"
server_key_version: "encryptionkey@0"
I20260619 02:29:48.156162 22082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "67a6e0a35fed4d5cacf97b11531c6e9a"
format_stamp: "Formatted at 2026-06-19 02:29:48 on dist-test-slave-4qw6"
server_key: "d6eaa605904af4954744d96caab322cb"
server_key_iv: "02bd0bb560656ef0ac69a5dbda0cb8d7"
server_key_version: "encryptionkey@0"
I20260619 02:29:48.160602 22082 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 02:29:48.164232 22096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:48.166046 22082 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 02:29:48.166224 22082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "67a6e0a35fed4d5cacf97b11531c6e9a"
format_stamp: "Formatted at 2026-06-19 02:29:48 on dist-test-slave-4qw6"
server_key: "d6eaa605904af4954744d96caab322cb"
server_key_iv: "02bd0bb560656ef0ac69a5dbda0cb8d7"
server_key_version: "encryptionkey@0"
I20260619 02:29:48.166378 22082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:48.187852 22082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:48.191855 22082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:48.192116 22082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:48.192854 22082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:48.194042 22082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:48.194123 22082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:48.194202 22082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:48.194262 22082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:48.204995 22082 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:37195
I20260619 02:29:48.205067 22209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:37195 every 8 connection(s)
I20260619 02:29:48.205917 22082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:29:48.214419 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 22082
I20260619 02:29:48.214596 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:29:48.214934 17795 external_mini_cluster.cc:1428] Setting key fcc08c2fba60debf6d6ef346809908e1
Jun 19 02:29:48 dist-test-slave-4qw6 krb5kdc[21843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836188, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:48.224387 21926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:48.208166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:36187 (local address 127.17.96.254:45725)
0619 02:29:48.208372 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:48.208377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:48.210065 (+ 1688us) server_negotiation.cc:407] Connection header received
0619 02:29:48.211216 (+ 1151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:48.211219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:48.211286 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:48.211406 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:48.213356 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.214002 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.215899 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.216138 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.216228 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:48.219981 (+ 3753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:48.219985 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:48.219987 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:48.220019 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:48.221629 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:48.222334 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:48.222339 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:48.222341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:48.222438 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:48.222894 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:48.222898 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:48.222900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:48.223134 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:48.223261 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:48.224031 (+ 770us) server_negotiation.cc:299] Negotiation successful
0619 02:29:48.224197 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260619 02:29:48.225442 22212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:48.208644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45725 (local address 127.17.96.194:36187)
0619 02:29:48.209807 (+ 1163us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:48.209876 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:48.210944 (+ 1068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:48.211682 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:48.211693 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:48.212250 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:48.212996 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:48.213114 (+ 118us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.214204 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.214209 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:48.215733 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:48.215748 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.216329 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.216336 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:48.216516 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:48.217365 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:48.217395 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:48.219729 (+ 2334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:48.221831 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:48.221840 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:48.221863 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:48.222204 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:48.222571 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:48.222578 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:48.222581 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:48.222774 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:48.223273 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:48.223284 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:48.223705 (+ 421us) client_negotiation.cc:769] Sending connection context
0619 02:29:48.223962 (+ 257us) client_negotiation.cc:240] Negotiation successful
0619 02:29:48.224380 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":128,"threads_started":1}
WARNING: no policy specified for kudu/127.17.96.195@KRBTEST.COM; defaulting to no policy
I20260619 02:29:48.227054 22210 heartbeater.cc:347] Connected to a master server at 127.17.96.254:45725
I20260619 02:29:48.227432 22210 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:48.228065 22210 heartbeater.cc:511] Master 127.17.96.254:45725 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.195@KRBTEST.COM" created.
I20260619 02:29:48.229715 21887 ts_manager.cc:194] Registered new tserver with Master: 67a6e0a35fed4d5cacf97b11531c6e9a (127.17.96.194:37195)
I20260619 02:29:48.230666 21887 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:36187
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
I20260619 02:29:48.245512 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45725
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:40549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:48.375485 22215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:48.375825 22215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:48.376124 22215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:48.382622 22215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:48.382711 22215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:48.382853 22215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:29:48.389678 22215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:40549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:48.391026 22215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:48.392196 22215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:48.403527 22221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:48.403690 22215 server_base.cc:1034] running on GCE node
W20260619 02:29:48.403513 22220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:48.403513 22223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:48.404413 22215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:48.405143 22215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:48.406322 22215 hybrid_clock.cc:648] HybridClock initialized: now 1781836188406283 us; error 61 us; skew 500 ppm
Jun 19 02:29:48 dist-test-slave-4qw6 krb5kdc[21843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836188, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:48.409605 22215 init.cc:377] Logged in from keytab as kudu/127.17.96.195@KRBTEST.COM (short username kudu)
I20260619 02:29:48.410581 22215 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:48.410660 22215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:48.410917 22215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:48.413475 22215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "452bc9f3dcf14c4ca39757cde12b9a8f"
format_stamp: "Formatted at 2026-06-19 02:29:48 on dist-test-slave-4qw6"
server_key: "040c90a70e51eac513fc17c1155aeae8"
server_key_iv: "0af85e65ec4adbb007eb19487bd91a38"
server_key_version: "encryptionkey@0"
I20260619 02:29:48.414296 22215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "452bc9f3dcf14c4ca39757cde12b9a8f"
format_stamp: "Formatted at 2026-06-19 02:29:48 on dist-test-slave-4qw6"
server_key: "040c90a70e51eac513fc17c1155aeae8"
server_key_iv: "0af85e65ec4adbb007eb19487bd91a38"
server_key_version: "encryptionkey@0"
I20260619 02:29:48.419214 22215 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 02:29:48.422616 22229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:48.424422 22215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:29:48.424608 22215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "452bc9f3dcf14c4ca39757cde12b9a8f"
format_stamp: "Formatted at 2026-06-19 02:29:48 on dist-test-slave-4qw6"
server_key: "040c90a70e51eac513fc17c1155aeae8"
server_key_iv: "0af85e65ec4adbb007eb19487bd91a38"
server_key_version: "encryptionkey@0"
I20260619 02:29:48.424757 22215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:48.435230 22215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:48.439853 22215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:48.440150 22215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:48.441118 22215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:48.442464 22215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:48.442554 22215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:48.442644 22215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:48.442708 22215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:48.455564 22215 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:39933
I20260619 02:29:48.455583 22342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:39933 every 8 connection(s)
I20260619 02:29:48.456435 22215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:29:48.464576 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 22215
I20260619 02:29:48.464757 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:29:48.465139 17795 external_mini_cluster.cc:1428] Setting key 2e26ba8d247bc0ef39d63deb3f70c0c2
Jun 19 02:29:48 dist-test-slave-4qw6 krb5kdc[21843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836188, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:48.473419 21926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:48.458833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:55079 (local address 127.17.96.254:45725)
0619 02:29:48.458994 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:48.458998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:48.460409 (+ 1411us) server_negotiation.cc:407] Connection header received
0619 02:29:48.461468 (+ 1059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:48.461472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:48.461540 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:48.461667 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:48.463162 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.463764 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.465253 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.465505 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.465640 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:48.468978 (+ 3338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:48.468982 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:48.468984 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:48.469015 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:48.470929 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:48.471728 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:48.471732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:48.471734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:48.471825 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:48.472173 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:48.472176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:48.472177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:48.472386 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:48.472471 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:48.473102 (+ 631us) server_negotiation.cc:299] Negotiation successful
0619 02:29:48.473268 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 02:29:48.474426 22345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:48.459246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45725 (local address 127.17.96.195:55079)
0619 02:29:48.460224 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:48.460277 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:48.461222 (+ 945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:48.461918 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:48.461926 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:48.462374 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:48.462986 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:48.463002 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.463929 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.463932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:48.465086 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:48.465100 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.465694 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.465700 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:48.466012 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:48.466904 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:48.466928 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:48.468788 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:48.471066 (+ 2278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:48.471072 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:48.471089 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:48.471608 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:48.471919 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:48.471921 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:48.471922 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:48.472082 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:48.472484 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:48.472490 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:48.472782 (+ 292us) client_negotiation.cc:769] Sending connection context
0619 02:29:48.472990 (+ 208us) client_negotiation.cc:240] Negotiation successful
0619 02:29:48.473305 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":145,"threads_started":1}
I20260619 02:29:48.475836 22343 heartbeater.cc:347] Connected to a master server at 127.17.96.254:45725
I20260619 02:29:48.476326 22343 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:48.477054 22343 heartbeater.cc:511] Master 127.17.96.254:45725 requested a full tablet report, sending...
I20260619 02:29:48.478838 21887 ts_manager.cc:194] Registered new tserver with Master: 452bc9f3dcf14c4ca39757cde12b9a8f (127.17.96.195:39933)
I20260619 02:29:48.479846 21887 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:55079
I20260619 02:29:48.480494 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:29:48.490674 21926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:48.482999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57800 (local address 127.17.96.254:45725)
0619 02:29:48.483160 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:48.483166 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:48.483330 (+ 164us) server_negotiation.cc:407] Connection header received
0619 02:29:48.483458 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:48.483460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:48.483507 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:48.483583 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:48.484298 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.484841 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.485680 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.485976 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.486064 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:48.487548 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:48.487551 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:48.487554 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:48.487572 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:48.488951 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:48.489365 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:48.489368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:48.489369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:48.489446 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:48.489678 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:48.489680 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:48.489681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:48.490012 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:48.490127 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:48.490348 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 02:29:48.490457 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 02:29:48.494228 21886 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:29:48.497845 21886 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:29:48.515324 22354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:48.508232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:39933 (local address 127.0.0.1:38002)
0619 02:29:48.508666 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:48.508717 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:48.508863 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:48.510384 (+ 1521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:48.510390 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:48.510411 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:48.510493 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:48.510497 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.511519 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.511526 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:48.513429 (+ 1903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:48.513440 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.514598 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.514604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:48.514764 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:48.514828 (+ 64us) client_negotiation.cc:769] Sending connection context
0619 02:29:48.514964 (+ 136us) client_negotiation.cc:240] Negotiation successful
0619 02:29:48.515119 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":222,"thread_start_us":103,"threads_started":1}
I20260619 02:29:48.515532 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:48.508875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45056 (local address 127.17.96.194:37195)
0619 02:29:48.509331 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:48.509348 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:48.509381 (+ 33us) server_negotiation.cc:407] Connection header received
0619 02:29:48.509541 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:48.509547 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:48.509810 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:48.510058 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:48.510400 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.511601 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.513531 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.514765 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.514930 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:48.515082 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:48.515209 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 02:29:48.515324 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":129,"threads_started":1}
I20260619 02:29:48.516294 22359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:48.509235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60956 (local address 127.17.96.193:32973)
0619 02:29:48.509632 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:48.509638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:48.510281 (+ 643us) server_negotiation.cc:407] Connection header received
0619 02:29:48.510377 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:48.510383 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:48.510571 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:48.510716 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:48.511116 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.512380 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.514174 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.515133 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.515242 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:48.515373 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:48.516055 (+ 682us) server_negotiation.cc:299] Negotiation successful
0619 02:29:48.516169 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":117,"threads_started":1}
I20260619 02:29:48.516325 22356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:48.508398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38002 (local address 127.17.96.195:39933)
0619 02:29:48.509689 (+ 1291us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:48.509704 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:48.509736 (+ 32us) server_negotiation.cc:407] Connection header received
0619 02:29:48.509870 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:48.509874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:48.510073 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:48.510226 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:48.510610 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.511369 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.513594 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.514443 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.514574 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:48.514738 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:48.515972 (+ 1234us) server_negotiation.cc:299] Negotiation successful
0619 02:29:48.516115 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1174,"thread_start_us":101,"threads_started":1}
I20260619 02:29:48.515504 22355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:48.508274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:37195 (local address 127.0.0.1:45056)
0619 02:29:48.509309 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:48.509323 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:48.509413 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:48.510067 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:48.510073 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:48.510099 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:48.510273 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:48.510280 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.511742 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.511746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:48.513346 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:48.513360 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.514929 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.514933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:48.515032 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:48.515058 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 02:29:48.515164 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 02:29:48.515257 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":968,"mutex_wait_us":93,"thread_start_us":109,"threads_started":1}
I20260619 02:29:48.515947 22358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:48.509056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:32973 (local address 127.0.0.1:60956)
0619 02:29:48.510058 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:48.510072 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:48.510176 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:48.510866 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:48.510869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:48.510887 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:48.510986 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:48.510991 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.512576 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.512580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:48.514005 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:48.514019 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.515467 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.515471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:48.515592 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:48.515615 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 02:29:48.515722 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 02:29:48.515805 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":933,"mutex_wait_us":50,"spinlock_wait_cycles":3328,"thread_start_us":86,"threads_started":1}
I20260619 02:29:48.519734 22277 tablet_service.cc:1467] Processing CreateTablet for tablet 316a7a24d6f74488af7f35acf968b199 (DEFAULT_TABLE table=test-table [id=89534df4e3dc4aff9f10cbc00e1834a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:48.519737 22144 tablet_service.cc:1467] Processing CreateTablet for tablet 316a7a24d6f74488af7f35acf968b199 (DEFAULT_TABLE table=test-table [id=89534df4e3dc4aff9f10cbc00e1834a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:48.521046 22144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 316a7a24d6f74488af7f35acf968b199. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:48.521054 22277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 316a7a24d6f74488af7f35acf968b199. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:48.523855 22011 tablet_service.cc:1467] Processing CreateTablet for tablet 316a7a24d6f74488af7f35acf968b199 (DEFAULT_TABLE table=test-table [id=89534df4e3dc4aff9f10cbc00e1834a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:48.525218 22011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 316a7a24d6f74488af7f35acf968b199. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:48.529539 22360 tablet_bootstrap.cc:492] T 316a7a24d6f74488af7f35acf968b199 P 67a6e0a35fed4d5cacf97b11531c6e9a: Bootstrap starting.
I20260619 02:29:48.530524 22361 tablet_bootstrap.cc:492] T 316a7a24d6f74488af7f35acf968b199 P 452bc9f3dcf14c4ca39757cde12b9a8f: Bootstrap starting.
I20260619 02:29:48.532655 22360 tablet_bootstrap.cc:654] T 316a7a24d6f74488af7f35acf968b199 P 67a6e0a35fed4d5cacf97b11531c6e9a: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:48.532863 22361 tablet_bootstrap.cc:654] T 316a7a24d6f74488af7f35acf968b199 P 452bc9f3dcf14c4ca39757cde12b9a8f: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:48.534349 22360 log.cc:824] T 316a7a24d6f74488af7f35acf968b199 P 67a6e0a35fed4d5cacf97b11531c6e9a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:48.534410 22361 log.cc:824] T 316a7a24d6f74488af7f35acf968b199 P 452bc9f3dcf14c4ca39757cde12b9a8f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:48.535055 22362 tablet_bootstrap.cc:492] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33: Bootstrap starting.
I20260619 02:29:48.537393 22360 tablet_bootstrap.cc:492] T 316a7a24d6f74488af7f35acf968b199 P 67a6e0a35fed4d5cacf97b11531c6e9a: No bootstrap required, opened a new log
I20260619 02:29:48.537673 22360 ts_tablet_manager.cc:1397] T 316a7a24d6f74488af7f35acf968b199 P 67a6e0a35fed4d5cacf97b11531c6e9a: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260619 02:29:48.537686 22361 tablet_bootstrap.cc:492] T 316a7a24d6f74488af7f35acf968b199 P 452bc9f3dcf14c4ca39757cde12b9a8f: No bootstrap required, opened a new log
I20260619 02:29:48.538021 22361 ts_tablet_manager.cc:1397] T 316a7a24d6f74488af7f35acf968b199 P 452bc9f3dcf14c4ca39757cde12b9a8f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260619 02:29:48.538214 22362 tablet_bootstrap.cc:654] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:48.539244 22362 log.cc:824] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:48.542099 22362 tablet_bootstrap.cc:492] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33: No bootstrap required, opened a new log
I20260619 02:29:48.542384 22362 ts_tablet_manager.cc:1397] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260619 02:29:48.542620 22360 raft_consensus.cc:348] T 316a7a24d6f74488af7f35acf968b199 P 67a6e0a35fed4d5cacf97b11531c6e9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452bc9f3dcf14c4ca39757cde12b9a8f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39933 } } peers { permanent_uuid: "67a6e0a35fed4d5cacf97b11531c6e9a" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37195 } } peers { permanent_uuid: "26a2b0e880394418b0b60071e082af33" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32973 } }
I20260619 02:29:48.542946 22360 raft_consensus.cc:374] T 316a7a24d6f74488af7f35acf968b199 P 67a6e0a35fed4d5cacf97b11531c6e9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:48.543004 22360 raft_consensus.cc:729] T 316a7a24d6f74488af7f35acf968b199 P 67a6e0a35fed4d5cacf97b11531c6e9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67a6e0a35fed4d5cacf97b11531c6e9a, State: Initialized, Role: FOLLOWER
I20260619 02:29:48.542759 22361 raft_consensus.cc:348] T 316a7a24d6f74488af7f35acf968b199 P 452bc9f3dcf14c4ca39757cde12b9a8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452bc9f3dcf14c4ca39757cde12b9a8f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39933 } } peers { permanent_uuid: "67a6e0a35fed4d5cacf97b11531c6e9a" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37195 } } peers { permanent_uuid: "26a2b0e880394418b0b60071e082af33" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32973 } }
I20260619 02:29:48.543463 22361 raft_consensus.cc:374] T 316a7a24d6f74488af7f35acf968b199 P 452bc9f3dcf14c4ca39757cde12b9a8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:48.543555 22361 raft_consensus.cc:729] T 316a7a24d6f74488af7f35acf968b199 P 452bc9f3dcf14c4ca39757cde12b9a8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452bc9f3dcf14c4ca39757cde12b9a8f, State: Initialized, Role: FOLLOWER
I20260619 02:29:48.543606 22360 consensus_queue.cc:260] T 316a7a24d6f74488af7f35acf968b199 P 67a6e0a35fed4d5cacf97b11531c6e9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452bc9f3dcf14c4ca39757cde12b9a8f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39933 } } peers { permanent_uuid: "67a6e0a35fed4d5cacf97b11531c6e9a" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37195 } } peers { permanent_uuid: "26a2b0e880394418b0b60071e082af33" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32973 } }
I20260619 02:29:48.544257 22361 consensus_queue.cc:260] T 316a7a24d6f74488af7f35acf968b199 P 452bc9f3dcf14c4ca39757cde12b9a8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452bc9f3dcf14c4ca39757cde12b9a8f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39933 } } peers { permanent_uuid: "67a6e0a35fed4d5cacf97b11531c6e9a" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37195 } } peers { permanent_uuid: "26a2b0e880394418b0b60071e082af33" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32973 } }
I20260619 02:29:48.544472 22210 heartbeater.cc:503] Master 127.17.96.254:45725 was elected leader, sending a full tablet report...
I20260619 02:29:48.545002 22360 ts_tablet_manager.cc:1428] T 316a7a24d6f74488af7f35acf968b199 P 67a6e0a35fed4d5cacf97b11531c6e9a: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260619 02:29:48.545801 22361 ts_tablet_manager.cc:1428] T 316a7a24d6f74488af7f35acf968b199 P 452bc9f3dcf14c4ca39757cde12b9a8f: Time spent starting tablet: real 0.008s user 0.002s sys 0.004s
I20260619 02:29:48.546353 22343 heartbeater.cc:503] Master 127.17.96.254:45725 was elected leader, sending a full tablet report...
I20260619 02:29:48.547439 22362 raft_consensus.cc:348] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452bc9f3dcf14c4ca39757cde12b9a8f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39933 } } peers { permanent_uuid: "67a6e0a35fed4d5cacf97b11531c6e9a" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37195 } } peers { permanent_uuid: "26a2b0e880394418b0b60071e082af33" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32973 } }
I20260619 02:29:48.547773 22362 raft_consensus.cc:374] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:48.547837 22362 raft_consensus.cc:729] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26a2b0e880394418b0b60071e082af33, State: Initialized, Role: FOLLOWER
I20260619 02:29:48.548403 22362 consensus_queue.cc:260] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452bc9f3dcf14c4ca39757cde12b9a8f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39933 } } peers { permanent_uuid: "67a6e0a35fed4d5cacf97b11531c6e9a" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37195 } } peers { permanent_uuid: "26a2b0e880394418b0b60071e082af33" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32973 } }
I20260619 02:29:48.549340 22077 heartbeater.cc:503] Master 127.17.96.254:45725 was elected leader, sending a full tablet report...
I20260619 02:29:48.549652 22362 ts_tablet_manager.cc:1428] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260619 02:29:48.581434 22368 raft_consensus.cc:482] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:29:48.581713 22368 raft_consensus.cc:504] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452bc9f3dcf14c4ca39757cde12b9a8f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39933 } } peers { permanent_uuid: "67a6e0a35fed4d5cacf97b11531c6e9a" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37195 } } peers { permanent_uuid: "26a2b0e880394418b0b60071e082af33" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32973 } }
I20260619 02:29:48.583160 22368 leader_election.cc:288] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452bc9f3dcf14c4ca39757cde12b9a8f (127.17.96.195:39933), 67a6e0a35fed4d5cacf97b11531c6e9a (127.17.96.194:37195)
I20260619 02:29:48.587610 22356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:48.583615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:60121 (local address 127.17.96.195:39933)
0619 02:29:48.583775 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:48.583780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:48.584066 (+ 286us) server_negotiation.cc:407] Connection header received
0619 02:29:48.584249 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:48.584252 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:48.584314 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:48.584409 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:48.584708 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.585249 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.586540 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.586970 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.587111 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:48.587169 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:48.587385 (+ 216us) server_negotiation.cc:299] Negotiation successful
0619 02:29:48.587462 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 02:29:48.587612 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:48.583718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:57613 (local address 127.17.96.194:37195)
0619 02:29:48.583862 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:48.583867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:48.584031 (+ 164us) server_negotiation.cc:407] Connection header received
0619 02:29:48.584221 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:48.584229 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:48.584276 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:48.584389 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:48.584719 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.585261 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.586548 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.586982 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.587120 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:48.587182 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:48.587405 (+ 223us) server_negotiation.cc:299] Negotiation successful
0619 02:29:48.587479 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:29:48.587637 22370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:48.583510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:39933 (local address 127.17.96.193:60121)
0619 02:29:48.583951 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:48.583964 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:48.584116 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:48.584430 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:48.584433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:48.584450 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:48.584588 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:48.584593 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.585376 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.585389 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:48.586399 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:48.586409 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.587106 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.587109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:48.587250 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:48.587270 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 02:29:48.587365 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 02:29:48.587462 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":90,"thread_start_us":137,"threads_started":1}
I20260619 02:29:48.587644 22369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:48.583490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:37195 (local address 127.17.96.193:57613)
0619 02:29:48.583904 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:48.583920 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:48.584086 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:48.584411 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:48.584414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:48.584431 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:48.584597 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:48.584603 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.585395 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.585399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:48.586405 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:48.586415 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.587115 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.587119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:48.587263 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:48.587290 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 02:29:48.587386 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 02:29:48.587478 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":28,"spinlock_wait_cycles":16256,"thread_start_us":137,"threads_started":1}
I20260619 02:29:48.588404 22164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "316a7a24d6f74488af7f35acf968b199" candidate_uuid: "26a2b0e880394418b0b60071e082af33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67a6e0a35fed4d5cacf97b11531c6e9a" is_pre_election: true
I20260619 02:29:48.588536 22297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "316a7a24d6f74488af7f35acf968b199" candidate_uuid: "26a2b0e880394418b0b60071e082af33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452bc9f3dcf14c4ca39757cde12b9a8f" is_pre_election: true
I20260619 02:29:48.588810 22297 raft_consensus.cc:2454] T 316a7a24d6f74488af7f35acf968b199 P 452bc9f3dcf14c4ca39757cde12b9a8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26a2b0e880394418b0b60071e082af33 in term 0.
I20260619 02:29:48.588810 22164 raft_consensus.cc:2454] T 316a7a24d6f74488af7f35acf968b199 P 67a6e0a35fed4d5cacf97b11531c6e9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26a2b0e880394418b0b60071e082af33 in term 0.
I20260619 02:29:48.589458 21964 leader_election.cc:302] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 26a2b0e880394418b0b60071e082af33, 452bc9f3dcf14c4ca39757cde12b9a8f, 67a6e0a35fed4d5cacf97b11531c6e9a; no voters:
I20260619 02:29:48.589847 22368 raft_consensus.cc:2781] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:29:48.589977 22368 raft_consensus.cc:482] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:29:48.590049 22368 raft_consensus.cc:3037] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:48.591398 22368 raft_consensus.cc:504] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452bc9f3dcf14c4ca39757cde12b9a8f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39933 } } peers { permanent_uuid: "67a6e0a35fed4d5cacf97b11531c6e9a" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37195 } } peers { permanent_uuid: "26a2b0e880394418b0b60071e082af33" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32973 } }
I20260619 02:29:48.591869 22368 leader_election.cc:288] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33 [CANDIDATE]: Term 1 election: Requested vote from peers 452bc9f3dcf14c4ca39757cde12b9a8f (127.17.96.195:39933), 67a6e0a35fed4d5cacf97b11531c6e9a (127.17.96.194:37195)
I20260619 02:29:48.592296 22297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "316a7a24d6f74488af7f35acf968b199" candidate_uuid: "26a2b0e880394418b0b60071e082af33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452bc9f3dcf14c4ca39757cde12b9a8f"
I20260619 02:29:48.592312 22164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "316a7a24d6f74488af7f35acf968b199" candidate_uuid: "26a2b0e880394418b0b60071e082af33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67a6e0a35fed4d5cacf97b11531c6e9a"
I20260619 02:29:48.592492 22164 raft_consensus.cc:3037] T 316a7a24d6f74488af7f35acf968b199 P 67a6e0a35fed4d5cacf97b11531c6e9a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:48.592494 22297 raft_consensus.cc:3037] T 316a7a24d6f74488af7f35acf968b199 P 452bc9f3dcf14c4ca39757cde12b9a8f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:48.593734 22297 raft_consensus.cc:2454] T 316a7a24d6f74488af7f35acf968b199 P 452bc9f3dcf14c4ca39757cde12b9a8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26a2b0e880394418b0b60071e082af33 in term 1.
I20260619 02:29:48.593736 22164 raft_consensus.cc:2454] T 316a7a24d6f74488af7f35acf968b199 P 67a6e0a35fed4d5cacf97b11531c6e9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26a2b0e880394418b0b60071e082af33 in term 1.
I20260619 02:29:48.594208 21964 leader_election.cc:302] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26a2b0e880394418b0b60071e082af33, 452bc9f3dcf14c4ca39757cde12b9a8f; no voters:
I20260619 02:29:48.594440 22368 raft_consensus.cc:2781] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:48.594749 22368 raft_consensus.cc:686] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33 [term 1 LEADER]: Becoming Leader. State: Replica: 26a2b0e880394418b0b60071e082af33, State: Running, Role: LEADER
I20260619 02:29:48.595288 22368 consensus_queue.cc:237] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452bc9f3dcf14c4ca39757cde12b9a8f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39933 } } peers { permanent_uuid: "67a6e0a35fed4d5cacf97b11531c6e9a" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37195 } } peers { permanent_uuid: "26a2b0e880394418b0b60071e082af33" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32973 } }
I20260619 02:29:48.600221 21887 catalog_manager.cc:5495] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33 reported cstate change: term changed from 0 to 1, leader changed from <none> to 26a2b0e880394418b0b60071e082af33 (127.17.96.193). New cstate: current_term: 1 leader_uuid: "26a2b0e880394418b0b60071e082af33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452bc9f3dcf14c4ca39757cde12b9a8f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67a6e0a35fed4d5cacf97b11531c6e9a" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26a2b0e880394418b0b60071e082af33" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32973 } health_report { overall_health: HEALTHY } } }
I20260619 02:29:48.620680 22359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:48.616633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60972 (local address 127.17.96.193:32973)
0619 02:29:48.616770 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:48.616775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:48.616798 (+ 23us) server_negotiation.cc:407] Connection header received
0619 02:29:48.616984 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:48.616987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:48.617047 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:48.617173 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:29:48.617477 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.618101 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.619036 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:48.619264 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:48.619380 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:48.619739 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:29:48.620223 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:29:48.620371 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:48.620447 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 02:29:48.620515 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 02:29:48.627759 22164 raft_consensus.cc:1261] T 316a7a24d6f74488af7f35acf968b199 P 67a6e0a35fed4d5cacf97b11531c6e9a [term 1 FOLLOWER]: Refusing update from remote peer 26a2b0e880394418b0b60071e082af33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:29:48.627759 22297 raft_consensus.cc:1261] T 316a7a24d6f74488af7f35acf968b199 P 452bc9f3dcf14c4ca39757cde12b9a8f [term 1 FOLLOWER]: Refusing update from remote peer 26a2b0e880394418b0b60071e082af33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:29:48.628568 22368 consensus_queue.cc:1035] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67a6e0a35fed4d5cacf97b11531c6e9a" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 37195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:29:48.628808 22371 consensus_queue.cc:1035] T 316a7a24d6f74488af7f35acf968b199 P 26a2b0e880394418b0b60071e082af33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452bc9f3dcf14c4ca39757cde12b9a8f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:29:48.637581 22376 mvcc.cc:205] Tried to move back new op lower bound from 7298401028610973696 to 7298401028490571776. Current Snapshot: MvccSnapshot[applied={T|T < 7298401028610973696}]
I20260619 02:29:48.648247 21886 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:29:48.648461 21886 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:29:48.651881 21886 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f99b9a7fc89a4ffd8b6ee01eee15b9a2: Sending DeleteTablet for 3 replicas of tablet 316a7a24d6f74488af7f35acf968b199
I20260619 02:29:48.653297 22144 tablet_service.cc:1514] Processing DeleteTablet for tablet 316a7a24d6f74488af7f35acf968b199 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:48 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:45056
I20260619 02:29:48.653394 22011 tablet_service.cc:1514] Processing DeleteTablet for tablet 316a7a24d6f74488af7f35acf968b199 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:48 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:60956
I20260619 02:29:48.654220 22386 tablet_replica.cc:330] T 316a7a24d6f74488af7f35acf968b199 P 67a6e0a35fed4d5cacf97b11531c6e9a: stopping tablet replica
I20260619 02:29:48.654362 22277 tablet_service.cc:1514] Processing DeleteTablet for tablet 316a7a24d6f74488af7f35acf968b199 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:48 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:38002
I20260619 02:29:48.654773 22386 raft_consensus.cc:2229] T 316a7a24d6f74488af7f35acf968b199 P 67a6e0a35fed4d5cacf97b11531c6e9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:29:48.655150 22387 tablet_replica.cc:330] T 316a7a24d6f74488af7f35acf968b199 P 452bc9f3dcf14c4ca39757cde12b9a8f: stopping tablet replica
I20260619 02:29:48.655231 22386 raft_consensus.cc:2258] T 316a7a24d6f74488af7f35acf968b199 P 67a6e0a35fed4d5cacf97b11531c6e9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:48.655645 22387 raft_consensus.cc:2229] T 316a7a24d6f74488af7f35acf968b199 P 452bc9f3dcf14c4ca39757cde12b9a8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:29:48.655869 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 21949
I20260619 02:29:48.656184 22387 raft_consensus.cc:2258] T 316a7a24d6f74488af7f35acf968b199 P 452bc9f3dcf14c4ca39757cde12b9a8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:48.658152 22387 ts_tablet_manager.cc:1905] T 316a7a24d6f74488af7f35acf968b199 P 452bc9f3dcf14c4ca39757cde12b9a8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:48.658154 22386 ts_tablet_manager.cc:1905] T 316a7a24d6f74488af7f35acf968b199 P 67a6e0a35fed4d5cacf97b11531c6e9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:48.661388 22387 ts_tablet_manager.cc:1918] T 316a7a24d6f74488af7f35acf968b199 P 452bc9f3dcf14c4ca39757cde12b9a8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:29:48.661573 22387 log.cc:1196] T 316a7a24d6f74488af7f35acf968b199 P 452bc9f3dcf14c4ca39757cde12b9a8f: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/316a7a24d6f74488af7f35acf968b199
I20260619 02:29:48.662002 22387 ts_tablet_manager.cc:1939] T 316a7a24d6f74488af7f35acf968b199 P 452bc9f3dcf14c4ca39757cde12b9a8f: Deleting consensus metadata
I20260619 02:29:48.662089 22386 ts_tablet_manager.cc:1918] T 316a7a24d6f74488af7f35acf968b199 P 67a6e0a35fed4d5cacf97b11531c6e9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:29:48.662217 22386 log.cc:1196] T 316a7a24d6f74488af7f35acf968b199 P 67a6e0a35fed4d5cacf97b11531c6e9a: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/316a7a24d6f74488af7f35acf968b199
I20260619 02:29:48.662554 22386 ts_tablet_manager.cc:1939] T 316a7a24d6f74488af7f35acf968b199 P 67a6e0a35fed4d5cacf97b11531c6e9a: Deleting consensus metadata
I20260619 02:29:48.663694 21871 catalog_manager.cc:4841] TS 452bc9f3dcf14c4ca39757cde12b9a8f (127.17.96.195:39933): tablet 316a7a24d6f74488af7f35acf968b199 (table test-table [id=89534df4e3dc4aff9f10cbc00e1834a8]) successfully deleted
I20260619 02:29:48.663890 21874 catalog_manager.cc:4841] TS 67a6e0a35fed4d5cacf97b11531c6e9a (127.17.96.194:37195): tablet 316a7a24d6f74488af7f35acf968b199 (table test-table [id=89534df4e3dc4aff9f10cbc00e1834a8]) successfully deleted
W20260619 02:29:48.665345 21873 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.96.193:32973 (error 108)
I20260619 02:29:48.666361 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 22082
I20260619 02:29:48.666682 22358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:48.666307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:32973 (local address 127.0.0.1:60976)
0619 02:29:48.666480 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:48.666568 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.193:32973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 02:29:48.667176 21873 catalog_manager.cc:4584] TS 26a2b0e880394418b0b60071e082af33 (127.17.96.193:32973): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 316a7a24d6f74488af7f35acf968b199: Network error: Client connection negotiation failed: client connection to 127.17.96.193:32973: connect: Connection refused (error 111)
I20260619 02:29:48.673261 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 22215
I20260619 02:29:48.680047 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 21856
2026-06-19T02:29:48Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2558 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-19T02:29:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:29:48Z Disabled control of system clock
I20260619 02:29:48.714351 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:46341
--webserver_interface=127.17.96.254
--webserver_port=0
--builtin_ntp_servers=127.17.96.212:40707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:46341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 02:29:48.840309 22392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:48.840706 22392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:48.840781 22392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:48.845389 22392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:29:48.845541 22392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:48.845573 22392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:48.845608 22392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:29:48.845649 22392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:29:48.851697 22392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:40707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:46341
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:46341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:48.853119 22392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:48.854771 22392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:48.864683 22397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:48.864636 22400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:48.864900 22392 server_base.cc:1034] running on GCE node
W20260619 02:29:48.864635 22398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:48.865726 22392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:48.867201 22392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:48.868408 22392 hybrid_clock.cc:648] HybridClock initialized: now 1781836188868391 us; error 38 us; skew 500 ppm
I20260619 02:29:48.870981 22392 webserver.cc:460] Webserver started at http://127.17.96.254:33495/ using document root <none> and password file <none>
I20260619 02:29:48.871723 22392 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:48.871812 22392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:48.872066 22392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:48.874611 22392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "0f95ed8c37c54e94b286a370f8ae4c5b"
format_stamp: "Formatted at 2026-06-19 02:29:48 on dist-test-slave-4qw6"
server_key: "163cfe5c4497ebb3315c46fccf93d537"
server_key_iv: "d6e51ab49a1d0030ceb9c4706701646d"
server_key_version: "encryptionkey@0"
I20260619 02:29:48.875358 22392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "0f95ed8c37c54e94b286a370f8ae4c5b"
format_stamp: "Formatted at 2026-06-19 02:29:48 on dist-test-slave-4qw6"
server_key: "163cfe5c4497ebb3315c46fccf93d537"
server_key_iv: "d6e51ab49a1d0030ceb9c4706701646d"
server_key_version: "encryptionkey@0"
I20260619 02:29:48.879853 22392 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:29:48.882961 22406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:48.884732 22392 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:29:48.884948 22392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "0f95ed8c37c54e94b286a370f8ae4c5b"
format_stamp: "Formatted at 2026-06-19 02:29:48 on dist-test-slave-4qw6"
server_key: "163cfe5c4497ebb3315c46fccf93d537"
server_key_iv: "d6e51ab49a1d0030ceb9c4706701646d"
server_key_version: "encryptionkey@0"
I20260619 02:29:48.885114 22392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:48.890563 22392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:48.891350 22392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:48.891582 22392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:48.899932 22392 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:46341
I20260619 02:29:48.899981 22458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:46341 every 8 connection(s)
I20260619 02:29:48.901271 22392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:29:48.902370 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 22392
I20260619 02:29:48.902518 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:29:48.902935 17795 external_mini_cluster.cc:1428] Setting key 3c16d4766ebdc1991b766cd6e5b9ff1d
I20260619 02:29:48.905797 22459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:48.910553 22462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:48.904678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50488 (local address 127.17.96.254:46341)
0619 02:29:48.906291 (+ 1613us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:48.906312 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:48.906360 (+ 48us) server_negotiation.cc:407] Connection header received
0619 02:29:48.907120 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:48.907139 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:48.907571 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:48.908008 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:48.908938 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:48.908943 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:48.908958 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:29:48.908970 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:48.909030 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:29:48.909048 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:48.909231 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:48.909315 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 02:29:48.909612 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1101,"thread_start_us":793,"threads_started":1}
I20260619 02:29:48.913609 22459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b: Bootstrap starting.
I20260619 02:29:48.916288 22459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:48.917348 22459 log.cc:824] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:48.919816 22459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b: No bootstrap required, opened a new log
I20260619 02:29:48.924504 22459 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f95ed8c37c54e94b286a370f8ae4c5b" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46341 } }
I20260619 02:29:48.924844 22459 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:48.924886 22459 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f95ed8c37c54e94b286a370f8ae4c5b, State: Initialized, Role: FOLLOWER
I20260619 02:29:48.925339 22459 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f95ed8c37c54e94b286a370f8ae4c5b" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46341 } }
I20260619 02:29:48.925472 22459 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:29:48.925521 22459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:29:48.925662 22459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:48.927186 22459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f95ed8c37c54e94b286a370f8ae4c5b" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46341 } }
I20260619 02:29:48.927644 22459 leader_election.cc:302] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f95ed8c37c54e94b286a370f8ae4c5b; no voters:
I20260619 02:29:48.928126 22459 leader_election.cc:288] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:29:48.928357 22464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:48.928747 22464 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b [term 1 LEADER]: Becoming Leader. State: Replica: 0f95ed8c37c54e94b286a370f8ae4c5b, State: Running, Role: LEADER
I20260619 02:29:48.929140 22464 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f95ed8c37c54e94b286a370f8ae4c5b" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46341 } }
I20260619 02:29:48.929653 22459 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:29:48.931149 22466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f95ed8c37c54e94b286a370f8ae4c5b. Latest consensus state: current_term: 1 leader_uuid: "0f95ed8c37c54e94b286a370f8ae4c5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f95ed8c37c54e94b286a370f8ae4c5b" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46341 } } }
I20260619 02:29:48.931339 22466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:48.931653 22465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f95ed8c37c54e94b286a370f8ae4c5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f95ed8c37c54e94b286a370f8ae4c5b" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46341 } } }
I20260619 02:29:48.931777 22465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:48.931752 22473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:29:48.936286 22473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:29:48.943862 22473 catalog_manager.cc:1347] Generated new cluster ID: 39acfc734b7841b68dd9a2617c249a78
I20260619 02:29:48.943972 22473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:29:48.975174 22473 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:29:48.975618 22473 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:29:48.980059 22473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b: Generated new TSK 0
I20260619 02:29:48.981089 22473 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:29:48.992084 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:46341
--builtin_ntp_servers=127.17.96.212:40707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 02:29:49.123553 22483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:49.123992 22483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:49.124101 22483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:49.129590 22483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:49.129698 22483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:49.129837 22483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:29:49.138478 22483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:40707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:46341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:49.140368 22483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:49.142570 22483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:49.153951 22491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:49.153951 22488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:49.154192 22483 server_base.cc:1034] running on GCE node
W20260619 02:29:49.153985 22489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:49.154876 22483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:49.155643 22483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:49.156838 22483 hybrid_clock.cc:648] HybridClock initialized: now 1781836189156818 us; error 39 us; skew 500 ppm
I20260619 02:29:49.159224 22483 webserver.cc:460] Webserver started at http://127.17.96.193:39719/ using document root <none> and password file <none>
I20260619 02:29:49.159901 22483 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:49.159984 22483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:49.160183 22483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:49.162768 22483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "ad7394e2d80645329ccefbe27fdfec70"
format_stamp: "Formatted at 2026-06-19 02:29:49 on dist-test-slave-4qw6"
server_key: "bfa27af1f18a731aee694816e1a9c2c2"
server_key_iv: "ca4f6af78d73f52b69be4f3e7922c9b2"
server_key_version: "encryptionkey@0"
I20260619 02:29:49.163419 22483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "ad7394e2d80645329ccefbe27fdfec70"
format_stamp: "Formatted at 2026-06-19 02:29:49 on dist-test-slave-4qw6"
server_key: "bfa27af1f18a731aee694816e1a9c2c2"
server_key_iv: "ca4f6af78d73f52b69be4f3e7922c9b2"
server_key_version: "encryptionkey@0"
I20260619 02:29:49.167882 22483 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260619 02:29:49.170872 22497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:49.172811 22483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:29:49.173077 22483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "ad7394e2d80645329ccefbe27fdfec70"
format_stamp: "Formatted at 2026-06-19 02:29:49 on dist-test-slave-4qw6"
server_key: "bfa27af1f18a731aee694816e1a9c2c2"
server_key_iv: "ca4f6af78d73f52b69be4f3e7922c9b2"
server_key_version: "encryptionkey@0"
I20260619 02:29:49.173262 22483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:49.178985 22483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:49.179915 22483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:49.180095 22483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:49.180788 22483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:49.181936 22483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:49.181989 22483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:49.182065 22483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:49.182099 22483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:49.191716 22483 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:37025
I20260619 02:29:49.191735 22610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:37025 every 8 connection(s)
I20260619 02:29:49.192804 22483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:29:49.199846 22462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:49.195020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:41811 (local address 127.17.96.254:46341)
0619 02:29:49.195183 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:49.195187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:49.196481 (+ 1294us) server_negotiation.cc:407] Connection header received
0619 02:29:49.197979 (+ 1498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:49.197982 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:49.198028 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:49.198117 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:49.199389 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:49.199392 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:49.199395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:29:49.199407 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:49.199429 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:29:49.199437 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:49.199527 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:49.199676 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 02:29:49.199719 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 02:29:49.200706 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 22483
I20260619 02:29:49.200865 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:29:49.200987 22613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:49.195477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:46341 (local address 127.17.96.193:41811)
0619 02:29:49.196277 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:49.196357 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:49.197681 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:49.198299 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:49.198306 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:49.198414 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:49.199057 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:29:49.199059 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:49.199120 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:29:49.199214 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:49.199525 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:49.199534 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:49.199540 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:29:49.199723 (+ 183us) client_negotiation.cc:240] Negotiation successful
0619 02:29:49.199996 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":132,"threads_started":1}
I20260619 02:29:49.201256 17795 external_mini_cluster.cc:1428] Setting key 958850dbdba05930c443623ccb83e8e8
I20260619 02:29:49.202502 22611 heartbeater.cc:347] Connected to a master server at 127.17.96.254:46341
I20260619 02:29:49.202811 22611 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:49.203321 22611 heartbeater.cc:511] Master 127.17.96.254:46341 requested a full tablet report, sending...
I20260619 02:29:49.203606 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:46341
--builtin_ntp_servers=127.17.96.212:40707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260619 02:29:49.205325 22421 ts_manager.cc:194] Registered new tserver with Master: ad7394e2d80645329ccefbe27fdfec70 (127.17.96.193:37025)
W20260619 02:29:49.327733 22614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:49.328080 22614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:49.328125 22614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:49.333103 22614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:49.333192 22614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:49.333259 22614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:29:49.338905 22614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:40707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:46341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:49.340175 22614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:49.341655 22614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:49.352559 22620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:49.352566 22622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:49.352552 22619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:49.352965 22614 server_base.cc:1034] running on GCE node
I20260619 02:29:49.353467 22614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:49.354316 22614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:49.355543 22614 hybrid_clock.cc:648] HybridClock initialized: now 1781836189355502 us; error 62 us; skew 500 ppm
I20260619 02:29:49.358084 22614 webserver.cc:460] Webserver started at http://127.17.96.194:40711/ using document root <none> and password file <none>
I20260619 02:29:49.358804 22614 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:49.358920 22614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:49.359181 22614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:49.361716 22614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "3cfcaf0c00bb404c947aab2a1c0bde48"
format_stamp: "Formatted at 2026-06-19 02:29:49 on dist-test-slave-4qw6"
server_key: "e0984c121da471e9b935a9956cd91b3b"
server_key_iv: "3f9dcbf7d2ebf84848d1a8dbe960f75d"
server_key_version: "encryptionkey@0"
I20260619 02:29:49.362499 22614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "3cfcaf0c00bb404c947aab2a1c0bde48"
format_stamp: "Formatted at 2026-06-19 02:29:49 on dist-test-slave-4qw6"
server_key: "e0984c121da471e9b935a9956cd91b3b"
server_key_iv: "3f9dcbf7d2ebf84848d1a8dbe960f75d"
server_key_version: "encryptionkey@0"
I20260619 02:29:49.367434 22614 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 02:29:49.371019 22628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:49.372910 22614 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260619 02:29:49.373123 22614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "3cfcaf0c00bb404c947aab2a1c0bde48"
format_stamp: "Formatted at 2026-06-19 02:29:49 on dist-test-slave-4qw6"
server_key: "e0984c121da471e9b935a9956cd91b3b"
server_key_iv: "3f9dcbf7d2ebf84848d1a8dbe960f75d"
server_key_version: "encryptionkey@0"
I20260619 02:29:49.373286 22614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:49.378990 22614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:49.379745 22614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:49.379962 22614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:49.380599 22614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:49.381670 22614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:49.381749 22614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:49.381881 22614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:49.381920 22614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:49.392608 22614 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:42351
I20260619 02:29:49.392623 22741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:42351 every 8 connection(s)
I20260619 02:29:49.393786 22614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:29:49.401345 22462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:49.396217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:49529 (local address 127.17.96.254:46341)
0619 02:29:49.396399 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:49.396403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:49.397738 (+ 1335us) server_negotiation.cc:407] Connection header received
0619 02:29:49.399292 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:49.399296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:49.399360 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:49.399455 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:49.400761 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:49.400764 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:49.400768 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:29:49.400783 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:49.400813 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:29:49.400824 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:49.400949 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:49.401109 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 02:29:49.401166 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 02:29:49.402544 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 22614
I20260619 02:29:49.402654 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:29:49.402678 22744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:49.396710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:46341 (local address 127.17.96.194:49529)
0619 02:29:49.397546 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:49.397597 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:49.399025 (+ 1428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:49.399675 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:49.399683 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:49.399789 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:49.400447 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:29:49.400449 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:49.400498 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:29:49.400580 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:49.400925 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:49.400935 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:49.400941 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:29:49.401168 (+ 227us) client_negotiation.cc:240] Negotiation successful
0619 02:29:49.401465 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":176,"threads_started":1}
I20260619 02:29:49.403064 17795 external_mini_cluster.cc:1428] Setting key cab26638378e5bc3931f83bf46f33111
I20260619 02:29:49.404253 22742 heartbeater.cc:347] Connected to a master server at 127.17.96.254:46341
I20260619 02:29:49.404560 22742 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:49.405047 22742 heartbeater.cc:511] Master 127.17.96.254:46341 requested a full tablet report, sending...
I20260619 02:29:49.405388 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:46341
--builtin_ntp_servers=127.17.96.212:40707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260619 02:29:49.406397 22421 ts_manager.cc:194] Registered new tserver with Master: 3cfcaf0c00bb404c947aab2a1c0bde48 (127.17.96.194:42351)
W20260619 02:29:49.533591 22745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:49.534039 22745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:49.534090 22745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:49.538658 22745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:49.538745 22745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:49.538822 22745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:29:49.544633 22745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:40707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:46341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:49.545933 22745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:49.547465 22745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:49.559003 22751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:49.559012 22750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:49.559041 22753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:49.559456 22745 server_base.cc:1034] running on GCE node
I20260619 02:29:49.560070 22745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:49.560904 22745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:49.562124 22745 hybrid_clock.cc:648] HybridClock initialized: now 1781836189562110 us; error 50 us; skew 500 ppm
I20260619 02:29:49.564514 22745 webserver.cc:460] Webserver started at http://127.17.96.195:36477/ using document root <none> and password file <none>
I20260619 02:29:49.565264 22745 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:49.565392 22745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:49.565667 22745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:49.568117 22745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "87869840c36e4418876c64c60299d6c1"
format_stamp: "Formatted at 2026-06-19 02:29:49 on dist-test-slave-4qw6"
server_key: "b663fe8fb238ee8df69cf66710fe0b7b"
server_key_iv: "772f08edce871c4b0de3014e606b988c"
server_key_version: "encryptionkey@0"
I20260619 02:29:49.568813 22745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "87869840c36e4418876c64c60299d6c1"
format_stamp: "Formatted at 2026-06-19 02:29:49 on dist-test-slave-4qw6"
server_key: "b663fe8fb238ee8df69cf66710fe0b7b"
server_key_iv: "772f08edce871c4b0de3014e606b988c"
server_key_version: "encryptionkey@0"
I20260619 02:29:49.573388 22745 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 02:29:49.577162 22759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:49.579212 22745 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260619 02:29:49.579383 22745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "87869840c36e4418876c64c60299d6c1"
format_stamp: "Formatted at 2026-06-19 02:29:49 on dist-test-slave-4qw6"
server_key: "b663fe8fb238ee8df69cf66710fe0b7b"
server_key_iv: "772f08edce871c4b0de3014e606b988c"
server_key_version: "encryptionkey@0"
I20260619 02:29:49.579519 22745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:49.584967 22745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:49.585660 22745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:49.585925 22745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:49.586519 22745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:49.587514 22745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:49.587563 22745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:49.587605 22745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:49.587621 22745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:49.597455 22745 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:46621
I20260619 02:29:49.597560 22872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:46621 every 8 connection(s)
I20260619 02:29:49.598579 22745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:29:49.603788 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 22745
I20260619 02:29:49.603912 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:29:49.604211 17795 external_mini_cluster.cc:1428] Setting key 9c49d4a59812c4a7dcb6dc4d3ad42151
I20260619 02:29:49.606720 22462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:49.601068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:44367 (local address 127.17.96.254:46341)
0619 02:29:49.601351 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:49.601357 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:49.602520 (+ 1163us) server_negotiation.cc:407] Connection header received
0619 02:29:49.604096 (+ 1576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:49.604100 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:49.604156 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:49.604257 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:49.605955 (+ 1698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:49.605959 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:49.605962 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:29:49.605984 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:49.606007 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:29:49.606016 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:49.606142 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:49.606442 (+ 300us) server_negotiation.cc:299] Negotiation successful
0619 02:29:49.606521 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260619 02:29:49.607791 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:49.601377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:46341 (local address 127.17.96.195:44367)
0619 02:29:49.602343 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:49.602393 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:49.603781 (+ 1388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:49.604478 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:49.604487 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:49.604595 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:49.605577 (+ 982us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:29:49.605580 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:49.605641 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:29:49.605724 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:49.606127 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:49.606137 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:49.606143 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:29:49.606371 (+ 228us) client_negotiation.cc:240] Negotiation successful
0619 02:29:49.606642 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":151,"threads_started":1}
I20260619 02:29:49.609531 22873 heartbeater.cc:347] Connected to a master server at 127.17.96.254:46341
I20260619 02:29:49.609947 22873 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:49.610440 22873 heartbeater.cc:511] Master 127.17.96.254:46341 requested a full tablet report, sending...
I20260619 02:29:49.611726 22421 ts_manager.cc:194] Registered new tserver with Master: 87869840c36e4418876c64c60299d6c1 (127.17.96.195:46621)
I20260619 02:29:49.620015 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:29:49.624637 22462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:49.622668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50494 (local address 127.17.96.254:46341)
0619 02:29:49.622910 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:49.622915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:49.622946 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:29:49.623252 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:49.623256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:49.623308 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:49.623400 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:49.624225 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:49.624229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:49.624231 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:29:49.624245 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:49.624266 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:29:49.624274 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:49.624359 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:49.624449 (+ 90us) server_negotiation.cc:299] Negotiation successful
0619 02:29:49.624495 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260619 02:29:49.628032 22421 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:29:49.631914 22421 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:29:49.647684 22885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:49.643512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:46621 (local address 127.0.0.1:41964)
0619 02:29:49.644505 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:49.644525 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:49.644689 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:49.645784 (+ 1095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:49.645787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:49.645837 (+ 50us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:49.646751 (+ 914us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:29:49.646753 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:49.646763 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:29:49.646782 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:49.647236 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:49.647247 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:49.647254 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 02:29:49.647374 (+ 120us) client_negotiation.cc:240] Negotiation successful
0619 02:29:49.647477 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":936,"spinlock_wait_cycles":9344,"thread_start_us":134,"threads_started":1}
I20260619 02:29:49.648110 22887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:49.643815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41964 (local address 127.17.96.195:46621)
0619 02:29:49.644334 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:49.644350 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:49.644661 (+ 311us) server_negotiation.cc:407] Connection header received
0619 02:29:49.644834 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:49.644841 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:49.645296 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:49.645508 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:49.646930 (+ 1422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:49.646937 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:49.646953 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:29:49.646966 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:49.647026 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:29:49.647065 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:49.647190 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:49.647805 (+ 615us) server_negotiation.cc:299] Negotiation successful
0619 02:29:49.647911 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":192,"threads_started":1}
I20260619 02:29:49.648499 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:49.645142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:42351 (local address 127.0.0.1:49062)
0619 02:29:49.645492 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:49.645507 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:49.645626 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:49.646947 (+ 1321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:49.646950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:49.646966 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:49.647462 (+ 496us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:29:49.647464 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:49.647476 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:29:49.647510 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:49.648115 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:49.648118 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:49.648122 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:29:49.648238 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 02:29:49.648298 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":124,"threads_started":1}
I20260619 02:29:49.648674 22889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:49.645353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49062 (local address 127.17.96.194:42351)
0619 02:29:49.645965 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:49.645981 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:49.646009 (+ 28us) server_negotiation.cc:407] Connection header received
0619 02:29:49.646073 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:49.646079 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:49.646259 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:49.646411 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:49.647739 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:49.647752 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:49.647771 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:29:49.647793 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:49.647863 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:29:49.647930 (+ 67us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:49.648065 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:49.648261 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 02:29:49.648348 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":181,"threads_started":1}
I20260619 02:29:49.649255 22884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:49.643364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:37025 (local address 127.0.0.1:34368)
0619 02:29:49.644457 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:49.644526 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:49.644689 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:49.645704 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:49.645714 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:49.645838 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:49.646627 (+ 789us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:29:49.646631 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:49.646658 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:29:49.646756 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:49.648995 (+ 2239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:49.648999 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:49.649003 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:29:49.649065 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 02:29:49.649108 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":867,"spinlock_wait_cycles":5120,"thread_start_us":111,"threads_started":1}
I20260619 02:29:49.650018 22886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:49.643521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34368 (local address 127.17.96.193:37025)
0619 02:29:49.645087 (+ 1566us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:49.645104 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:49.645137 (+ 33us) server_negotiation.cc:407] Connection header received
0619 02:29:49.645207 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:49.645216 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:49.645493 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:49.646224 (+ 731us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:49.647793 (+ 1569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:49.647799 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:49.647808 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:29:49.647818 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:49.648826 (+ 1008us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:29:49.648877 (+ 51us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:49.649627 (+ 750us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:49.649697 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 02:29:49.649849 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1394,"thread_start_us":147,"threads_started":1}
I20260619 02:29:49.652065 22807 tablet_service.cc:1467] Processing CreateTablet for tablet 4c127202ac5f41149ffe998a252f29a6 (DEFAULT_TABLE table=test-table [id=3348feeb618d4a01b76cd3ac1b89c443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:49.653261 22807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c127202ac5f41149ffe998a252f29a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:49.653235 22676 tablet_service.cc:1467] Processing CreateTablet for tablet 4c127202ac5f41149ffe998a252f29a6 (DEFAULT_TABLE table=test-table [id=3348feeb618d4a01b76cd3ac1b89c443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:49.653235 22545 tablet_service.cc:1467] Processing CreateTablet for tablet 4c127202ac5f41149ffe998a252f29a6 (DEFAULT_TABLE table=test-table [id=3348feeb618d4a01b76cd3ac1b89c443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:49.654434 22545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c127202ac5f41149ffe998a252f29a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:49.654434 22676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c127202ac5f41149ffe998a252f29a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:49.661062 22890 tablet_bootstrap.cc:492] T 4c127202ac5f41149ffe998a252f29a6 P 87869840c36e4418876c64c60299d6c1: Bootstrap starting.
I20260619 02:29:49.663479 22891 tablet_bootstrap.cc:492] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70: Bootstrap starting.
I20260619 02:29:49.663708 22890 tablet_bootstrap.cc:654] T 4c127202ac5f41149ffe998a252f29a6 P 87869840c36e4418876c64c60299d6c1: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:49.664746 22890 log.cc:824] T 4c127202ac5f41149ffe998a252f29a6 P 87869840c36e4418876c64c60299d6c1: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:49.664754 22892 tablet_bootstrap.cc:492] T 4c127202ac5f41149ffe998a252f29a6 P 3cfcaf0c00bb404c947aab2a1c0bde48: Bootstrap starting.
I20260619 02:29:49.666567 22891 tablet_bootstrap.cc:654] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:49.667580 22891 log.cc:824] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:49.667634 22892 tablet_bootstrap.cc:654] T 4c127202ac5f41149ffe998a252f29a6 P 3cfcaf0c00bb404c947aab2a1c0bde48: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:49.667735 22890 tablet_bootstrap.cc:492] T 4c127202ac5f41149ffe998a252f29a6 P 87869840c36e4418876c64c60299d6c1: No bootstrap required, opened a new log
I20260619 02:29:49.668013 22890 ts_tablet_manager.cc:1397] T 4c127202ac5f41149ffe998a252f29a6 P 87869840c36e4418876c64c60299d6c1: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260619 02:29:49.668661 22892 log.cc:824] T 4c127202ac5f41149ffe998a252f29a6 P 3cfcaf0c00bb404c947aab2a1c0bde48: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:49.670428 22891 tablet_bootstrap.cc:492] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70: No bootstrap required, opened a new log
I20260619 02:29:49.670712 22891 ts_tablet_manager.cc:1397] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260619 02:29:49.671265 22892 tablet_bootstrap.cc:492] T 4c127202ac5f41149ffe998a252f29a6 P 3cfcaf0c00bb404c947aab2a1c0bde48: No bootstrap required, opened a new log
I20260619 02:29:49.671516 22892 ts_tablet_manager.cc:1397] T 4c127202ac5f41149ffe998a252f29a6 P 3cfcaf0c00bb404c947aab2a1c0bde48: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260619 02:29:49.673393 22890 raft_consensus.cc:348] T 4c127202ac5f41149ffe998a252f29a6 P 87869840c36e4418876c64c60299d6c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7394e2d80645329ccefbe27fdfec70" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37025 } } peers { permanent_uuid: "87869840c36e4418876c64c60299d6c1" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46621 } } peers { permanent_uuid: "3cfcaf0c00bb404c947aab2a1c0bde48" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42351 } }
I20260619 02:29:49.673812 22890 raft_consensus.cc:374] T 4c127202ac5f41149ffe998a252f29a6 P 87869840c36e4418876c64c60299d6c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:49.673874 22890 raft_consensus.cc:729] T 4c127202ac5f41149ffe998a252f29a6 P 87869840c36e4418876c64c60299d6c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87869840c36e4418876c64c60299d6c1, State: Initialized, Role: FOLLOWER
I20260619 02:29:49.674532 22890 consensus_queue.cc:260] T 4c127202ac5f41149ffe998a252f29a6 P 87869840c36e4418876c64c60299d6c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7394e2d80645329ccefbe27fdfec70" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37025 } } peers { permanent_uuid: "87869840c36e4418876c64c60299d6c1" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46621 } } peers { permanent_uuid: "3cfcaf0c00bb404c947aab2a1c0bde48" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42351 } }
I20260619 02:29:49.675570 22873 heartbeater.cc:503] Master 127.17.96.254:46341 was elected leader, sending a full tablet report...
I20260619 02:29:49.676085 22890 ts_tablet_manager.cc:1428] T 4c127202ac5f41149ffe998a252f29a6 P 87869840c36e4418876c64c60299d6c1: Time spent starting tablet: real 0.008s user 0.004s sys 0.005s
I20260619 02:29:49.675848 22891 raft_consensus.cc:348] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7394e2d80645329ccefbe27fdfec70" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37025 } } peers { permanent_uuid: "87869840c36e4418876c64c60299d6c1" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46621 } } peers { permanent_uuid: "3cfcaf0c00bb404c947aab2a1c0bde48" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42351 } }
I20260619 02:29:49.676191 22891 raft_consensus.cc:374] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:49.676268 22891 raft_consensus.cc:729] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad7394e2d80645329ccefbe27fdfec70, State: Initialized, Role: FOLLOWER
I20260619 02:29:49.676610 22892 raft_consensus.cc:348] T 4c127202ac5f41149ffe998a252f29a6 P 3cfcaf0c00bb404c947aab2a1c0bde48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7394e2d80645329ccefbe27fdfec70" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37025 } } peers { permanent_uuid: "87869840c36e4418876c64c60299d6c1" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46621 } } peers { permanent_uuid: "3cfcaf0c00bb404c947aab2a1c0bde48" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42351 } }
I20260619 02:29:49.676869 22892 raft_consensus.cc:374] T 4c127202ac5f41149ffe998a252f29a6 P 3cfcaf0c00bb404c947aab2a1c0bde48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:49.676937 22892 raft_consensus.cc:729] T 4c127202ac5f41149ffe998a252f29a6 P 3cfcaf0c00bb404c947aab2a1c0bde48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cfcaf0c00bb404c947aab2a1c0bde48, State: Initialized, Role: FOLLOWER
I20260619 02:29:49.676837 22891 consensus_queue.cc:260] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7394e2d80645329ccefbe27fdfec70" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37025 } } peers { permanent_uuid: "87869840c36e4418876c64c60299d6c1" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46621 } } peers { permanent_uuid: "3cfcaf0c00bb404c947aab2a1c0bde48" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42351 } }
I20260619 02:29:49.677438 22892 consensus_queue.cc:260] T 4c127202ac5f41149ffe998a252f29a6 P 3cfcaf0c00bb404c947aab2a1c0bde48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7394e2d80645329ccefbe27fdfec70" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37025 } } peers { permanent_uuid: "87869840c36e4418876c64c60299d6c1" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46621 } } peers { permanent_uuid: "3cfcaf0c00bb404c947aab2a1c0bde48" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42351 } }
I20260619 02:29:49.678325 22891 ts_tablet_manager.cc:1428] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260619 02:29:49.678608 22892 ts_tablet_manager.cc:1428] T 4c127202ac5f41149ffe998a252f29a6 P 3cfcaf0c00bb404c947aab2a1c0bde48: Time spent starting tablet: real 0.007s user 0.002s sys 0.006s
I20260619 02:29:49.678871 22611 heartbeater.cc:503] Master 127.17.96.254:46341 was elected leader, sending a full tablet report...
I20260619 02:29:49.679143 22742 heartbeater.cc:503] Master 127.17.96.254:46341 was elected leader, sending a full tablet report...
W20260619 02:29:49.694186 22612 tablet.cc:2369] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:29:49.744563 22897 raft_consensus.cc:482] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:29:49.744822 22897 raft_consensus.cc:504] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7394e2d80645329ccefbe27fdfec70" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37025 } } peers { permanent_uuid: "87869840c36e4418876c64c60299d6c1" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46621 } } peers { permanent_uuid: "3cfcaf0c00bb404c947aab2a1c0bde48" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42351 } }
I20260619 02:29:49.746297 22897 leader_election.cc:288] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87869840c36e4418876c64c60299d6c1 (127.17.96.195:46621), 3cfcaf0c00bb404c947aab2a1c0bde48 (127.17.96.194:42351)
I20260619 02:29:49.749413 22887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:49.746872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:37403 (local address 127.17.96.195:46621)
0619 02:29:49.747097 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:49.747103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:49.747128 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:29:49.747222 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:49.747226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:49.747307 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:49.747458 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:49.748664 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:49.748669 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:49.748675 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:29:49.748700 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:49.748738 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:29:49.748753 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:49.748879 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:49.749100 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 02:29:49.749179 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 02:29:49.749399 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:49.746568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:46621 (local address 127.17.96.193:37403)
0619 02:29:49.746914 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:49.746934 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:49.747037 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:49.747618 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:49.747622 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:49.747646 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:49.748405 (+ 759us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:29:49.748408 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:49.748424 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:29:49.748462 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:49.748882 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:49.748889 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:49.748897 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 02:29:49.749079 (+ 182us) client_negotiation.cc:240] Negotiation successful
0619 02:29:49.749145 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":138,"threads_started":1}
I20260619 02:29:49.749562 22889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:49.747336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:37735 (local address 127.17.96.194:42351)
0619 02:29:49.747501 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:49.747507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:49.747826 (+ 319us) server_negotiation.cc:407] Connection header received
0619 02:29:49.747974 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:49.747979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:49.748038 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:49.748158 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:49.749048 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:49.749054 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:49.749057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:29:49.749070 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:49.749093 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:29:49.749104 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:49.749202 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:49.749327 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 02:29:49.749394 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 02:29:49.749939 22900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:49.747090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:42351 (local address 127.17.96.193:37735)
0619 02:29:49.747681 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:49.747703 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:49.747828 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:49.748269 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:49.748274 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:49.748294 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:49.748855 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:29:49.748857 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:49.748869 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:29:49.748896 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:49.749206 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:49.749210 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:49.749213 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 02:29:49.749313 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 02:29:49.749361 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":163,"threads_started":1}
I20260619 02:29:49.750571 22827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c127202ac5f41149ffe998a252f29a6" candidate_uuid: "ad7394e2d80645329ccefbe27fdfec70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87869840c36e4418876c64c60299d6c1" is_pre_election: true
I20260619 02:29:49.750703 22696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c127202ac5f41149ffe998a252f29a6" candidate_uuid: "ad7394e2d80645329ccefbe27fdfec70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cfcaf0c00bb404c947aab2a1c0bde48" is_pre_election: true
I20260619 02:29:49.751041 22827 raft_consensus.cc:2454] T 4c127202ac5f41149ffe998a252f29a6 P 87869840c36e4418876c64c60299d6c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad7394e2d80645329ccefbe27fdfec70 in term 0.
I20260619 02:29:49.751041 22696 raft_consensus.cc:2454] T 4c127202ac5f41149ffe998a252f29a6 P 3cfcaf0c00bb404c947aab2a1c0bde48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad7394e2d80645329ccefbe27fdfec70 in term 0.
I20260619 02:29:49.751718 22499 leader_election.cc:302] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cfcaf0c00bb404c947aab2a1c0bde48, ad7394e2d80645329ccefbe27fdfec70; no voters:
I20260619 02:29:49.752187 22897 raft_consensus.cc:2781] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:29:49.752300 22897 raft_consensus.cc:482] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:29:49.752336 22897 raft_consensus.cc:3037] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:49.753547 22897 raft_consensus.cc:504] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7394e2d80645329ccefbe27fdfec70" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37025 } } peers { permanent_uuid: "87869840c36e4418876c64c60299d6c1" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46621 } } peers { permanent_uuid: "3cfcaf0c00bb404c947aab2a1c0bde48" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42351 } }
I20260619 02:29:49.754045 22897 leader_election.cc:288] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70 [CANDIDATE]: Term 1 election: Requested vote from peers 87869840c36e4418876c64c60299d6c1 (127.17.96.195:46621), 3cfcaf0c00bb404c947aab2a1c0bde48 (127.17.96.194:42351)
I20260619 02:29:49.754846 22827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c127202ac5f41149ffe998a252f29a6" candidate_uuid: "ad7394e2d80645329ccefbe27fdfec70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87869840c36e4418876c64c60299d6c1"
I20260619 02:29:49.754875 22696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c127202ac5f41149ffe998a252f29a6" candidate_uuid: "ad7394e2d80645329ccefbe27fdfec70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cfcaf0c00bb404c947aab2a1c0bde48"
I20260619 02:29:49.755064 22827 raft_consensus.cc:3037] T 4c127202ac5f41149ffe998a252f29a6 P 87869840c36e4418876c64c60299d6c1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:49.755085 22696 raft_consensus.cc:3037] T 4c127202ac5f41149ffe998a252f29a6 P 3cfcaf0c00bb404c947aab2a1c0bde48 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:49.756356 22696 raft_consensus.cc:2454] T 4c127202ac5f41149ffe998a252f29a6 P 3cfcaf0c00bb404c947aab2a1c0bde48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad7394e2d80645329ccefbe27fdfec70 in term 1.
I20260619 02:29:49.756361 22827 raft_consensus.cc:2454] T 4c127202ac5f41149ffe998a252f29a6 P 87869840c36e4418876c64c60299d6c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad7394e2d80645329ccefbe27fdfec70 in term 1.
I20260619 02:29:49.756764 22499 leader_election.cc:302] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cfcaf0c00bb404c947aab2a1c0bde48, ad7394e2d80645329ccefbe27fdfec70; no voters:
I20260619 02:29:49.756991 22897 raft_consensus.cc:2781] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:49.757285 22897 raft_consensus.cc:686] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70 [term 1 LEADER]: Becoming Leader. State: Replica: ad7394e2d80645329ccefbe27fdfec70, State: Running, Role: LEADER
I20260619 02:29:49.757711 22897 consensus_queue.cc:237] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7394e2d80645329ccefbe27fdfec70" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37025 } } peers { permanent_uuid: "87869840c36e4418876c64c60299d6c1" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46621 } } peers { permanent_uuid: "3cfcaf0c00bb404c947aab2a1c0bde48" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42351 } }
I20260619 02:29:49.762449 22421 catalog_manager.cc:5495] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad7394e2d80645329ccefbe27fdfec70 (127.17.96.193). New cstate: current_term: 1 leader_uuid: "ad7394e2d80645329ccefbe27fdfec70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad7394e2d80645329ccefbe27fdfec70" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87869840c36e4418876c64c60299d6c1" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3cfcaf0c00bb404c947aab2a1c0bde48" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42351 } health_report { overall_health: UNKNOWN } } }
I20260619 02:29:49.775513 22886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:49.773723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34384 (local address 127.17.96.193:37025)
0619 02:29:49.773923 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:49.773930 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:49.773950 (+ 20us) server_negotiation.cc:407] Connection header received
0619 02:29:49.774059 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:49.774087 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:49.774159 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:49.774274 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:49.775013 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:49.775017 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:49.775023 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:29:49.775040 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:49.775060 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:29:49.775082 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:49.775186 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:49.775294 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 02:29:49.775344 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260619 02:29:49.782693 22827 raft_consensus.cc:1261] T 4c127202ac5f41149ffe998a252f29a6 P 87869840c36e4418876c64c60299d6c1 [term 1 FOLLOWER]: Refusing update from remote peer ad7394e2d80645329ccefbe27fdfec70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:29:49.782693 22696 raft_consensus.cc:1261] T 4c127202ac5f41149ffe998a252f29a6 P 3cfcaf0c00bb404c947aab2a1c0bde48 [term 1 FOLLOWER]: Refusing update from remote peer ad7394e2d80645329ccefbe27fdfec70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:29:49.783586 22901 consensus_queue.cc:1035] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cfcaf0c00bb404c947aab2a1c0bde48" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:29:49.783844 22897 consensus_queue.cc:1035] T 4c127202ac5f41149ffe998a252f29a6 P ad7394e2d80645329ccefbe27fdfec70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87869840c36e4418876c64c60299d6c1" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:29:49.793535 22906 mvcc.cc:205] Tried to move back new op lower bound from 7298401033340862464 to 7298401033251987456. Current Snapshot: MvccSnapshot[applied={T|T < 7298401033340862464}]
I20260619 02:29:49.802269 22421 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:29:49.802479 22421 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:29:49.805227 22421 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f95ed8c37c54e94b286a370f8ae4c5b: Sending DeleteTablet for 3 replicas of tablet 4c127202ac5f41149ffe998a252f29a6
I20260619 02:29:49.806249 22545 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c127202ac5f41149ffe998a252f29a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:49 UTC) from {username='slave'} at 127.0.0.1:34368
I20260619 02:29:49.806586 22807 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c127202ac5f41149ffe998a252f29a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:49 UTC) from {username='slave'} at 127.0.0.1:41964
I20260619 02:29:49.806600 22676 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c127202ac5f41149ffe998a252f29a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:49 UTC) from {username='slave'} at 127.0.0.1:49062
I20260619 02:29:49.807128 22915 tablet_replica.cc:330] T 4c127202ac5f41149ffe998a252f29a6 P 3cfcaf0c00bb404c947aab2a1c0bde48: stopping tablet replica
I20260619 02:29:49.807256 22916 tablet_replica.cc:330] T 4c127202ac5f41149ffe998a252f29a6 P 87869840c36e4418876c64c60299d6c1: stopping tablet replica
I20260619 02:29:49.807528 22915 raft_consensus.cc:2229] T 4c127202ac5f41149ffe998a252f29a6 P 3cfcaf0c00bb404c947aab2a1c0bde48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:29:49.807572 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 22483
I20260619 02:29:49.808457 22916 raft_consensus.cc:2229] T 4c127202ac5f41149ffe998a252f29a6 P 87869840c36e4418876c64c60299d6c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:29:49.808840 22916 raft_consensus.cc:2258] T 4c127202ac5f41149ffe998a252f29a6 P 87869840c36e4418876c64c60299d6c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:49.809271 22915 raft_consensus.cc:2258] T 4c127202ac5f41149ffe998a252f29a6 P 3cfcaf0c00bb404c947aab2a1c0bde48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:49.810652 22916 ts_tablet_manager.cc:1905] T 4c127202ac5f41149ffe998a252f29a6 P 87869840c36e4418876c64c60299d6c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:49.811005 22915 ts_tablet_manager.cc:1905] T 4c127202ac5f41149ffe998a252f29a6 P 3cfcaf0c00bb404c947aab2a1c0bde48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:49.813159 22916 ts_tablet_manager.cc:1918] T 4c127202ac5f41149ffe998a252f29a6 P 87869840c36e4418876c64c60299d6c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:29:49.813316 22916 log.cc:1196] T 4c127202ac5f41149ffe998a252f29a6 P 87869840c36e4418876c64c60299d6c1: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/4c127202ac5f41149ffe998a252f29a6
I20260619 02:29:49.813634 22916 ts_tablet_manager.cc:1939] T 4c127202ac5f41149ffe998a252f29a6 P 87869840c36e4418876c64c60299d6c1: Deleting consensus metadata
I20260619 02:29:49.814919 22915 ts_tablet_manager.cc:1918] T 4c127202ac5f41149ffe998a252f29a6 P 3cfcaf0c00bb404c947aab2a1c0bde48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:29:49.815047 22915 log.cc:1196] T 4c127202ac5f41149ffe998a252f29a6 P 3cfcaf0c00bb404c947aab2a1c0bde48: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/4c127202ac5f41149ffe998a252f29a6
I20260619 02:29:49.815055 22408 catalog_manager.cc:4841] TS 87869840c36e4418876c64c60299d6c1 (127.17.96.195:46621): tablet 4c127202ac5f41149ffe998a252f29a6 (table test-table [id=3348feeb618d4a01b76cd3ac1b89c443]) successfully deleted
I20260619 02:29:49.815414 22915 ts_tablet_manager.cc:1939] T 4c127202ac5f41149ffe998a252f29a6 P 3cfcaf0c00bb404c947aab2a1c0bde48: Deleting consensus metadata
W20260619 02:29:49.815857 22410 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.96.193:37025 (error 108)
I20260619 02:29:49.816123 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 22614
I20260619 02:29:49.818291 22884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:49.817513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:37025 (local address 127.0.0.1:34398)
0619 02:29:49.817915 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:49.818152 (+ 237us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.193:37025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251}
W20260619 02:29:49.818529 22410 catalog_manager.cc:4584] TS ad7394e2d80645329ccefbe27fdfec70 (127.17.96.193:37025): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4c127202ac5f41149ffe998a252f29a6: Network error: Client connection negotiation failed: client connection to 127.17.96.193:37025: connect: Connection refused (error 111)
I20260619 02:29:49.825192 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 22745
I20260619 02:29:49.826442 22884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:49.825464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:42351 (local address 127.0.0.1:49078)
0619 02:29:49.826295 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:49.826351 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.194:42351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":736}
I20260619 02:29:49.832156 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 22392
2026-06-19T02:29:49Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1152 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260619 02:29:49.856803 17795 mini_oidc.cc:152] Starting JWKS server
I20260619 02:29:49.859781 17795 webserver.cc:460] Webserver started at https://127.0.0.1:44455/ using document root <none> and password file <none>
I20260619 02:29:49.860064 17795 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 02:29:49.860466 17795 webserver.cc:460] Webserver started at http://127.0.0.1:39173/ using document root <none> and password file <none>
2026-06-19T02:29:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:29:49Z Disabled control of system clock
I20260619 02:29:49.872342 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:43419
--webserver_interface=127.17.96.254
--webserver_port=0
--builtin_ntp_servers=127.17.96.212:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:43419
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44455/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836162001484-17795-0/testchainca.cert with env {}
W20260619 02:29:50.010100 22923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:50.010438 22923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:50.010485 22923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:50.015511 22923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:29:50.015615 22923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:50.015637 22923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:50.015659 22923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:29:50.015679 22923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 02:29:50.015698 22923 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 02:29:50.015714 22923 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44455/jwks.json
I20260619 02:29:50.022643 22923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:43419
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:43419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44455/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836162001484-17795-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:50.023926 22923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:50.025075 22923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:50.035542 22928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:50.035554 22929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:50.035856 22923 server_base.cc:1034] running on GCE node
W20260619 02:29:50.035511 22931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:50.036983 22923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:50.038709 22923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:50.039971 22923 hybrid_clock.cc:648] HybridClock initialized: now 1781836190039942 us; error 55 us; skew 500 ppm
I20260619 02:29:50.042663 22923 webserver.cc:460] Webserver started at http://127.17.96.254:43783/ using document root <none> and password file <none>
I20260619 02:29:50.043435 22923 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:50.043511 22923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:50.043787 22923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:50.046329 22923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "2e5c604d573d405eafcef4b0b1ea20d7"
format_stamp: "Formatted at 2026-06-19 02:29:50 on dist-test-slave-4qw6"
server_key: "b4a1742a48e810420a33cd8908b789b3"
server_key_iv: "21bdd83de009e783ed1f397e76f56ad9"
server_key_version: "encryptionkey@0"
I20260619 02:29:50.047012 22923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "2e5c604d573d405eafcef4b0b1ea20d7"
format_stamp: "Formatted at 2026-06-19 02:29:50 on dist-test-slave-4qw6"
server_key: "b4a1742a48e810420a33cd8908b789b3"
server_key_iv: "21bdd83de009e783ed1f397e76f56ad9"
server_key_version: "encryptionkey@0"
I20260619 02:29:50.051626 22923 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:29:50.054885 22937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:50.056416 22923 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:29:50.056617 22923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "2e5c604d573d405eafcef4b0b1ea20d7"
format_stamp: "Formatted at 2026-06-19 02:29:50 on dist-test-slave-4qw6"
server_key: "b4a1742a48e810420a33cd8908b789b3"
server_key_iv: "21bdd83de009e783ed1f397e76f56ad9"
server_key_version: "encryptionkey@0"
I20260619 02:29:50.056768 22923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:50.076717 22923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:50.077658 22923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:50.077957 22923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:50.086442 22923 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:43419
I20260619 02:29:50.086449 22989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:43419 every 8 connection(s)
I20260619 02:29:50.087703 22923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:29:50.090694 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 22923
I20260619 02:29:50.090817 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:29:50.091153 17795 external_mini_cluster.cc:1428] Setting key 9e8b5e0062c23a682019e7a3229da399
I20260619 02:29:50.092849 22990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:50.102160 22990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7: Bootstrap starting.
I20260619 02:29:50.104928 22993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:50.093178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38934 (local address 127.17.96.254:43419)
0619 02:29:50.094167 (+ 989us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:50.094192 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:50.094256 (+ 64us) server_negotiation.cc:407] Connection header received
0619 02:29:50.095416 (+ 1160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:50.095471 (+ 55us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:50.096196 (+ 725us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:50.096724 (+ 528us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:50.097293 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:50.098952 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:50.100111 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:50.100909 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:50.101089 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:50.102376 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:50.102384 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:50.102402 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:29:50.102982 (+ 580us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:50.103064 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:29:50.103096 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:50.103214 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:50.103469 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 02:29:50.103830 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":164,"threads_started":1}
I20260619 02:29:50.105474 22990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:50.106410 22990 log.cc:824] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:50.108824 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:29:50.108872 22990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7: No bootstrap required, opened a new log
I20260619 02:29:50.113358 22990 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e5c604d573d405eafcef4b0b1ea20d7" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 43419 } }
I20260619 02:29:50.113656 22990 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:50.113698 22990 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e5c604d573d405eafcef4b0b1ea20d7, State: Initialized, Role: FOLLOWER
I20260619 02:29:50.114288 22990 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e5c604d573d405eafcef4b0b1ea20d7" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 43419 } }
I20260619 02:29:50.114496 22990 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:29:50.114601 22990 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:29:50.114745 22990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:50.115893 22990 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e5c604d573d405eafcef4b0b1ea20d7" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 43419 } }
I20260619 02:29:50.116326 22990 leader_election.cc:302] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e5c604d573d405eafcef4b0b1ea20d7; no voters:
I20260619 02:29:50.116716 22990 leader_election.cc:288] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:29:50.116951 22995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:50.117363 22995 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7 [term 1 LEADER]: Becoming Leader. State: Replica: 2e5c604d573d405eafcef4b0b1ea20d7, State: Running, Role: LEADER
I20260619 02:29:50.117880 22995 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e5c604d573d405eafcef4b0b1ea20d7" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 43419 } }
I20260619 02:29:50.118455 22990 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:29:50.120317 22996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e5c604d573d405eafcef4b0b1ea20d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e5c604d573d405eafcef4b0b1ea20d7" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 43419 } } }
I20260619 02:29:50.120425 22997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e5c604d573d405eafcef4b0b1ea20d7. Latest consensus state: current_term: 1 leader_uuid: "2e5c604d573d405eafcef4b0b1ea20d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e5c604d573d405eafcef4b0b1ea20d7" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 43419 } } }
I20260619 02:29:50.120539 22996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:50.120575 22997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7 [sys.catalog]: This master's current role is: LEADER
W20260619 02:29:50.124577 23010 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:29:50.124708 23010 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:29:50.124826 23008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:29:50.125680 23008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:29:50.132190 23008 catalog_manager.cc:1347] Generated new cluster ID: e8a04b9aeacf44d49413a9a2302392f1
I20260619 02:29:50.132296 23008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:29:50.161970 23008 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:29:50.163188 23008 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:29:50.168969 23008 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e5c604d573d405eafcef4b0b1ea20d7: Generated new TSK 0
I20260619 02:29:50.169862 23008 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 02:29:50.206481 23020 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260619 02:29:50.206670 23020 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 02:29:50.207262 22993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:50.187309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38938 (local address 127.17.96.254:43419)
0619 02:29:50.187461 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:50.187467 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:50.187643 (+ 176us) server_negotiation.cc:407] Connection header received
0619 02:29:50.187890 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:50.187893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:50.187963 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:50.188089 (+ 126us) server_negotiation.cc:226] Negotiated authn=JWT
0619 02:29:50.188352 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:50.189205 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:50.190245 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:50.190625 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:50.190744 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:50.191115 (+ 371us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 02:29:50.206762 (+ 15647us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44455/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 02:29:50.207095 (+ 333us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38938: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44455/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":64}
W20260619 02:29:50.207373 22993 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38938: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44455/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260619 02:29:50.208659 23019 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:29:50.187195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:43419 (local address 127.0.0.1:38938)
0619 02:29:50.187545 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:50.187559 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:50.187694 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:50.188116 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:50.188118 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:50.188125 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0619 02:29:50.188244 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:50.188248 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:50.189349 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:50.189351 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:50.190113 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:50.190123 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:50.190756 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:50.190759 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:50.190828 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:50.190883 (+ 55us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 02:29:50.207089 (+ 16206us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44455/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 02:29:50.207414 (+ 325us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.96.254:43419: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44455/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":117,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.17.96.254:43419: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44455/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260619 02:29:50.209435 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 22923
2026-06-19T02:29:50Z chronyd exiting
I20260619 02:29:50.248422 17795 test_util.cc:175] -----------------------------------------------
I20260619 02:29:50.248525 17795 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781836162001484-17795-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (393 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260619 02:29:50.249644 17795 mini_oidc.cc:152] Starting JWKS server
I20260619 02:29:50.250643 17795 webserver.cc:460] Webserver started at https://127.0.0.1:43607/ using document root <none> and password file <none>
I20260619 02:29:50.250730 17795 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 02:29:50.250949 17795 webserver.cc:460] Webserver started at http://127.0.0.1:46665/ using document root <none> and password file <none>
2026-06-19T02:29:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:29:50Z Disabled control of system clock
I20260619 02:29:50.263248 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:40691
--webserver_interface=127.17.96.254
--webserver_port=0
--builtin_ntp_servers=127.17.96.212:37563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:40691
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43607/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836162001484-17795-0/testchainca.cert with env {}
W20260619 02:29:50.401664 23026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:50.402180 23026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:50.402238 23026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:50.408267 23026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:29:50.408382 23026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:50.408401 23026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:50.408422 23026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:29:50.408440 23026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 02:29:50.408461 23026 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 02:29:50.408476 23026 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43607/jwks.json
I20260619 02:29:50.415367 23026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:37563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:40691
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:40691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43607/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836162001484-17795-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:50.416679 23026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:50.417881 23026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:50.427712 23032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:50.427712 23031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:50.427739 23034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:50.428496 23026 server_base.cc:1034] running on GCE node
I20260619 02:29:50.429507 23026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:50.431389 23026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:50.432700 23026 hybrid_clock.cc:648] HybridClock initialized: now 1781836190432649 us; error 72 us; skew 500 ppm
I20260619 02:29:50.435397 23026 webserver.cc:460] Webserver started at http://127.17.96.254:40535/ using document root <none> and password file <none>
I20260619 02:29:50.436161 23026 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:50.436278 23026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:50.436542 23026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:50.439153 23026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "8296676992ec421bbdb9460e226ff658"
format_stamp: "Formatted at 2026-06-19 02:29:50 on dist-test-slave-4qw6"
server_key: "8a33fca908108207b4c1dca1142b9bae"
server_key_iv: "fbeeb6ce847dde631a1634ae031940af"
server_key_version: "encryptionkey@0"
I20260619 02:29:50.439870 23026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "8296676992ec421bbdb9460e226ff658"
format_stamp: "Formatted at 2026-06-19 02:29:50 on dist-test-slave-4qw6"
server_key: "8a33fca908108207b4c1dca1142b9bae"
server_key_iv: "fbeeb6ce847dde631a1634ae031940af"
server_key_version: "encryptionkey@0"
I20260619 02:29:50.444448 23026 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:29:50.447502 23040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:50.449071 23026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:29:50.449273 23026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "8296676992ec421bbdb9460e226ff658"
format_stamp: "Formatted at 2026-06-19 02:29:50 on dist-test-slave-4qw6"
server_key: "8a33fca908108207b4c1dca1142b9bae"
server_key_iv: "fbeeb6ce847dde631a1634ae031940af"
server_key_version: "encryptionkey@0"
I20260619 02:29:50.449429 23026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:50.466125 23026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:50.466974 23026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:50.467201 23026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:50.475492 23026 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:40691
I20260619 02:29:50.475492 23092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:40691 every 8 connection(s)
I20260619 02:29:50.476712 23026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:29:50.481547 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 23026
I20260619 02:29:50.481612 23093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:50.481706 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:29:50.482115 17795 external_mini_cluster.cc:1428] Setting key a019d683223aa82d9eebf68b3e01b184
I20260619 02:29:50.490693 23093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8296676992ec421bbdb9460e226ff658: Bootstrap starting.
I20260619 02:29:50.493894 23093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8296676992ec421bbdb9460e226ff658: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:50.494817 23096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:50.483762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33176 (local address 127.17.96.254:40691)
0619 02:29:50.484567 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:50.484590 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:50.484642 (+ 52us) server_negotiation.cc:407] Connection header received
0619 02:29:50.485434 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:50.485462 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:50.486103 (+ 641us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:50.486540 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:50.487078 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:50.488595 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:50.489977 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:50.490770 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:50.490979 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:50.492165 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:50.492172 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:50.492191 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:29:50.492763 (+ 572us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:50.492850 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:29:50.492881 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:50.492997 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:50.493270 (+ 273us) server_negotiation.cc:299] Negotiation successful
0619 02:29:50.493595 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":141,"threads_started":1}
I20260619 02:29:50.494980 23093 log.cc:824] T 00000000000000000000000000000000 P 8296676992ec421bbdb9460e226ff658: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:50.497922 23093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8296676992ec421bbdb9460e226ff658: No bootstrap required, opened a new log
I20260619 02:29:50.498704 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:29:50.502519 23093 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8296676992ec421bbdb9460e226ff658 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8296676992ec421bbdb9460e226ff658" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40691 } }
I20260619 02:29:50.502810 23093 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8296676992ec421bbdb9460e226ff658 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:50.502880 23093 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8296676992ec421bbdb9460e226ff658 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8296676992ec421bbdb9460e226ff658, State: Initialized, Role: FOLLOWER
I20260619 02:29:50.503417 23093 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8296676992ec421bbdb9460e226ff658 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8296676992ec421bbdb9460e226ff658" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40691 } }
I20260619 02:29:50.503597 23093 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8296676992ec421bbdb9460e226ff658 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:29:50.503703 23093 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8296676992ec421bbdb9460e226ff658 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:29:50.503834 23093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8296676992ec421bbdb9460e226ff658 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:50.504931 23093 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8296676992ec421bbdb9460e226ff658 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8296676992ec421bbdb9460e226ff658" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40691 } }
I20260619 02:29:50.505340 23093 leader_election.cc:302] T 00000000000000000000000000000000 P 8296676992ec421bbdb9460e226ff658 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8296676992ec421bbdb9460e226ff658; no voters:
I20260619 02:29:50.505724 23093 leader_election.cc:288] T 00000000000000000000000000000000 P 8296676992ec421bbdb9460e226ff658 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:29:50.505955 23098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8296676992ec421bbdb9460e226ff658 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:50.506240 23098 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8296676992ec421bbdb9460e226ff658 [term 1 LEADER]: Becoming Leader. State: Replica: 8296676992ec421bbdb9460e226ff658, State: Running, Role: LEADER
I20260619 02:29:50.506610 23098 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8296676992ec421bbdb9460e226ff658 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8296676992ec421bbdb9460e226ff658" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40691 } }
I20260619 02:29:50.507217 23093 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8296676992ec421bbdb9460e226ff658 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:29:50.508644 23099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8296676992ec421bbdb9460e226ff658 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8296676992ec421bbdb9460e226ff658" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8296676992ec421bbdb9460e226ff658" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40691 } } }
I20260619 02:29:50.508689 23100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8296676992ec421bbdb9460e226ff658 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8296676992ec421bbdb9460e226ff658. Latest consensus state: current_term: 1 leader_uuid: "8296676992ec421bbdb9460e226ff658" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8296676992ec421bbdb9460e226ff658" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40691 } } }
I20260619 02:29:50.508922 23100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8296676992ec421bbdb9460e226ff658 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:50.508876 23099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8296676992ec421bbdb9460e226ff658 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:50.510202 23109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:29:50.514039 23109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:29:50.521373 23109 catalog_manager.cc:1347] Generated new cluster ID: 3d24327200584c879ad3ebe28cc2286d
I20260619 02:29:50.521507 23109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:29:50.529464 23109 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:29:50.530691 23109 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:29:50.539001 23109 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8296676992ec421bbdb9460e226ff658: Generated new TSK 0
I20260619 02:29:50.540024 23109 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 02:29:50.576664 23123 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260619 02:29:50.576786 23123 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 02:29:50.577541 23096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:50.559822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33184 (local address 127.17.96.254:40691)
0619 02:29:50.560056 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:50.560063 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:50.560407 (+ 344us) server_negotiation.cc:407] Connection header received
0619 02:29:50.560590 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:50.560593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:50.560663 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:50.560791 (+ 128us) server_negotiation.cc:226] Negotiated authn=JWT
0619 02:29:50.561109 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:50.561715 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:50.563220 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:50.563459 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:50.563550 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:50.563953 (+ 403us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 02:29:50.577141 (+ 13188us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43607/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 02:29:50.577413 (+ 272us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33184: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43607/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":92}
W20260619 02:29:50.577641 23096 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33184: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43607/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260619 02:29:50.577880 23122 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:29:50.559690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40691 (local address 127.0.0.1:33184)
0619 02:29:50.560191 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:50.560212 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:50.560419 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:50.560819 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:50.560823 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:50.560833 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0619 02:29:50.560972 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:50.560980 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:50.561927 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:50.561931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:50.563066 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:50.563078 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:50.563595 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:50.563600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:50.563717 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:50.563752 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 02:29:50.577389 (+ 13637us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43607/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 02:29:50.577636 (+ 247us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.96.254:40691: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43607/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":184,"threads_started":1}
I20260619 02:29:50.578541 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 23026
2026-06-19T02:29:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (368 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260619 02:29:50.618381 17795 mini_oidc.cc:152] Starting JWKS server
I20260619 02:29:50.619292 17795 webserver.cc:460] Webserver started at https://127.0.0.1:40649/ using document root <none> and password file <none>
I20260619 02:29:50.619378 17795 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 02:29:50.619593 17795 webserver.cc:460] Webserver started at http://127.0.0.1:42911/ using document root <none> and password file <none>
2026-06-19T02:29:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:29:50Z Disabled control of system clock
I20260619 02:29:50.630822 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:42445
--webserver_interface=127.17.96.254
--webserver_port=0
--builtin_ntp_servers=127.17.96.212:37547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:42445
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40649/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 02:29:50.755448 23129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:50.755807 23129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:50.755887 23129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:50.760648 23129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:29:50.760741 23129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:50.760759 23129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:50.760778 23129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:29:50.760797 23129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 02:29:50.760814 23129 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 02:29:50.760871 23129 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40649/jwks.json
I20260619 02:29:50.766464 23129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:37547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:42445
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:42445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40649/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:50.767820 23129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:50.769425 23129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:50.779659 23134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:50.779659 23135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:50.779702 23137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:50.780134 23129 server_base.cc:1034] running on GCE node
I20260619 02:29:50.780839 23129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:50.782406 23129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:50.783757 23129 hybrid_clock.cc:648] HybridClock initialized: now 1781836190783720 us; error 77 us; skew 500 ppm
I20260619 02:29:50.786363 23129 webserver.cc:460] Webserver started at http://127.17.96.254:36407/ using document root <none> and password file <none>
I20260619 02:29:50.787086 23129 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:50.787210 23129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:50.787542 23129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:50.790212 23129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "ade8ae40bf2f4c19b7f7361c8c7504b8"
format_stamp: "Formatted at 2026-06-19 02:29:50 on dist-test-slave-4qw6"
server_key: "0243388249a5862952abf7564ea215fa"
server_key_iv: "cc6f5fe0d18f0353b903dcaa1c99f3b1"
server_key_version: "encryptionkey@0"
I20260619 02:29:50.790967 23129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "ade8ae40bf2f4c19b7f7361c8c7504b8"
format_stamp: "Formatted at 2026-06-19 02:29:50 on dist-test-slave-4qw6"
server_key: "0243388249a5862952abf7564ea215fa"
server_key_iv: "cc6f5fe0d18f0353b903dcaa1c99f3b1"
server_key_version: "encryptionkey@0"
I20260619 02:29:50.795614 23129 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:29:50.798919 23143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:50.800836 23129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 02:29:50.801039 23129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "ade8ae40bf2f4c19b7f7361c8c7504b8"
format_stamp: "Formatted at 2026-06-19 02:29:50 on dist-test-slave-4qw6"
server_key: "0243388249a5862952abf7564ea215fa"
server_key_iv: "cc6f5fe0d18f0353b903dcaa1c99f3b1"
server_key_version: "encryptionkey@0"
I20260619 02:29:50.801200 23129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:50.818892 23129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:50.819792 23129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:50.820055 23129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:50.828012 23129 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:42445
I20260619 02:29:50.828019 23195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:42445 every 8 connection(s)
I20260619 02:29:50.829231 23129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:29:50.833115 23196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:50.838698 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 23129
I20260619 02:29:50.838826 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:29:50.839159 17795 external_mini_cluster.cc:1428] Setting key 286912a8638fac037881dd7c64883fd0
I20260619 02:29:50.840006 23196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ade8ae40bf2f4c19b7f7361c8c7504b8: Bootstrap starting.
I20260619 02:29:50.842739 23196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ade8ae40bf2f4c19b7f7361c8c7504b8: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:50.843780 23196 log.cc:824] T 00000000000000000000000000000000 P ade8ae40bf2f4c19b7f7361c8c7504b8: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:50.847136 23196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ade8ae40bf2f4c19b7f7361c8c7504b8: No bootstrap required, opened a new log
I20260619 02:29:50.850889 23199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:50.840881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41644 (local address 127.17.96.254:42445)
0619 02:29:50.841649 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:50.841666 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:50.841708 (+ 42us) server_negotiation.cc:407] Connection header received
0619 02:29:50.842544 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:50.842564 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:50.843104 (+ 540us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:50.843438 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:50.843965 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:50.845397 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:50.846878 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:50.847540 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:50.847763 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:50.848804 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:50.848810 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:50.848826 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:29:50.849303 (+ 477us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:50.849365 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:29:50.849388 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:50.849517 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:50.849651 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 02:29:50.850022 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":175,"threads_started":1}
I20260619 02:29:50.851498 23196 raft_consensus.cc:348] T 00000000000000000000000000000000 P ade8ae40bf2f4c19b7f7361c8c7504b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade8ae40bf2f4c19b7f7361c8c7504b8" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42445 } }
I20260619 02:29:50.851802 23196 raft_consensus.cc:374] T 00000000000000000000000000000000 P ade8ae40bf2f4c19b7f7361c8c7504b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:50.851895 23196 raft_consensus.cc:729] T 00000000000000000000000000000000 P ade8ae40bf2f4c19b7f7361c8c7504b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ade8ae40bf2f4c19b7f7361c8c7504b8, State: Initialized, Role: FOLLOWER
I20260619 02:29:50.852448 23196 consensus_queue.cc:260] T 00000000000000000000000000000000 P ade8ae40bf2f4c19b7f7361c8c7504b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade8ae40bf2f4c19b7f7361c8c7504b8" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42445 } }
I20260619 02:29:50.852627 23196 raft_consensus.cc:388] T 00000000000000000000000000000000 P ade8ae40bf2f4c19b7f7361c8c7504b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:29:50.852730 23196 raft_consensus.cc:482] T 00000000000000000000000000000000 P ade8ae40bf2f4c19b7f7361c8c7504b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:29:50.852859 23196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ade8ae40bf2f4c19b7f7361c8c7504b8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:50.854238 23196 raft_consensus.cc:504] T 00000000000000000000000000000000 P ade8ae40bf2f4c19b7f7361c8c7504b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade8ae40bf2f4c19b7f7361c8c7504b8" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42445 } }
I20260619 02:29:50.854305 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:29:50.854653 23196 leader_election.cc:302] T 00000000000000000000000000000000 P ade8ae40bf2f4c19b7f7361c8c7504b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ade8ae40bf2f4c19b7f7361c8c7504b8; no voters:
I20260619 02:29:50.855062 23196 leader_election.cc:288] T 00000000000000000000000000000000 P ade8ae40bf2f4c19b7f7361c8c7504b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:29:50.855144 23201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ade8ae40bf2f4c19b7f7361c8c7504b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:50.855353 23201 raft_consensus.cc:686] T 00000000000000000000000000000000 P ade8ae40bf2f4c19b7f7361c8c7504b8 [term 1 LEADER]: Becoming Leader. State: Replica: ade8ae40bf2f4c19b7f7361c8c7504b8, State: Running, Role: LEADER
I20260619 02:29:50.855686 23201 consensus_queue.cc:237] T 00000000000000000000000000000000 P ade8ae40bf2f4c19b7f7361c8c7504b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade8ae40bf2f4c19b7f7361c8c7504b8" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42445 } }
I20260619 02:29:50.856514 23196 sys_catalog.cc:564] T 00000000000000000000000000000000 P ade8ae40bf2f4c19b7f7361c8c7504b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:29:50.857638 23203 sys_catalog.cc:455] T 00000000000000000000000000000000 P ade8ae40bf2f4c19b7f7361c8c7504b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ade8ae40bf2f4c19b7f7361c8c7504b8. Latest consensus state: current_term: 1 leader_uuid: "ade8ae40bf2f4c19b7f7361c8c7504b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade8ae40bf2f4c19b7f7361c8c7504b8" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42445 } } }
I20260619 02:29:50.857689 23202 sys_catalog.cc:455] T 00000000000000000000000000000000 P ade8ae40bf2f4c19b7f7361c8c7504b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ade8ae40bf2f4c19b7f7361c8c7504b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade8ae40bf2f4c19b7f7361c8c7504b8" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42445 } } }
I20260619 02:29:50.857838 23203 sys_catalog.cc:458] T 00000000000000000000000000000000 P ade8ae40bf2f4c19b7f7361c8c7504b8 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:50.857838 23202 sys_catalog.cc:458] T 00000000000000000000000000000000 P ade8ae40bf2f4c19b7f7361c8c7504b8 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:50.859073 23210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:29:50.862283 23210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:29:50.870034 23210 catalog_manager.cc:1347] Generated new cluster ID: 9a33570ba7a84ce7b47e06974c343f64
I20260619 02:29:50.870174 23210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:29:50.903796 23210 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:29:50.905344 23210 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:29:50.916594 23210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ade8ae40bf2f4c19b7f7361c8c7504b8: Generated new TSK 0
I20260619 02:29:50.917536 23210 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:29:50.941670 23199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:50.934743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41660 (local address 127.17.96.254:42445)
0619 02:29:50.934902 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:50.934908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:50.935093 (+ 185us) server_negotiation.cc:407] Connection header received
0619 02:29:50.935262 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:50.935266 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:50.935337 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:50.935492 (+ 155us) server_negotiation.cc:226] Negotiated authn=JWT
0619 02:29:50.935727 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:50.936619 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:50.937652 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:50.938086 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:50.938205 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:50.938576 (+ 371us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 02:29:50.941082 (+ 2506us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40649/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 02:29:50.941458 (+ 376us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41660: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40649/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":53}
W20260619 02:29:50.941838 23199 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41660: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40649/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260619 02:29:50.943424 23225 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:29:50.934562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:42445 (local address 127.0.0.1:41660)
0619 02:29:50.934991 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:50.935006 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:50.935121 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:50.935490 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:50.935492 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:50.935499 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0619 02:29:50.935612 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:50.935616 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:50.936791 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:50.936794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:50.937504 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:50.937512 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:50.938251 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:50.938255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:50.938331 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:50.938360 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 02:29:50.941349 (+ 2989us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40649/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 02:29:50.943211 (+ 1862us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.96.254:42445: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40649/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":1543,"thread_start_us":132,"threads_started":1}
I20260619 02:29:50.944499 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 23129
E20260619 02:29:50.953701 23226 webserver.cc:533] Webserver: sslize failed:
E20260619 02:29:50.953881 23226 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-19T02:29:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (367 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-19T02:29:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:29:50Z Disabled control of system clock
I20260619 02:29:50.996475 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:39687
--webserver_interface=127.17.96.254
--webserver_port=0
--builtin_ntp_servers=127.17.96.212:35973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:39687
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836162001484-17795-0/testchainca.cert with env {}
W20260619 02:29:51.121378 23230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:51.121721 23230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:51.121798 23230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:51.126536 23230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:29:51.126627 23230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:51.126645 23230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:51.126664 23230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:29:51.126724 23230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 02:29:51.126770 23230 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 02:29:51.126791 23230 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260619 02:29:51.134164 23230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:35973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:39687
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:39687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836162001484-17795-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:51.135485 23230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:51.137295 23230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:51.150520 23238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:51.150628 23230 server_base.cc:1034] running on GCE node
W20260619 02:29:51.150542 23236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:51.150550 23235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:51.151605 23230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:51.152966 23230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:51.156284 23230 hybrid_clock.cc:648] HybridClock initialized: now 1781836191156227 us; error 147 us; skew 500 ppm
I20260619 02:29:51.158928 23230 webserver.cc:460] Webserver started at http://127.17.96.254:38865/ using document root <none> and password file <none>
I20260619 02:29:51.159665 23230 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:51.159749 23230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:51.160000 23230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:51.162493 23230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "74158b5b78fd4cd3bad1eed0d8a0c91b"
format_stamp: "Formatted at 2026-06-19 02:29:51 on dist-test-slave-4qw6"
server_key: "3b29ff2591d4c51f2e1f9648f1c1ca94"
server_key_iv: "657b71ccce65fee2dd40cb8101bb4c3e"
server_key_version: "encryptionkey@0"
I20260619 02:29:51.163167 23230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "74158b5b78fd4cd3bad1eed0d8a0c91b"
format_stamp: "Formatted at 2026-06-19 02:29:51 on dist-test-slave-4qw6"
server_key: "3b29ff2591d4c51f2e1f9648f1c1ca94"
server_key_iv: "657b71ccce65fee2dd40cb8101bb4c3e"
server_key_version: "encryptionkey@0"
I20260619 02:29:51.167915 23230 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:29:51.171945 23244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:51.173888 23230 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 02:29:51.174088 23230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "74158b5b78fd4cd3bad1eed0d8a0c91b"
format_stamp: "Formatted at 2026-06-19 02:29:51 on dist-test-slave-4qw6"
server_key: "3b29ff2591d4c51f2e1f9648f1c1ca94"
server_key_iv: "657b71ccce65fee2dd40cb8101bb4c3e"
server_key_version: "encryptionkey@0"
I20260619 02:29:51.174259 23230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:51.212483 23230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:51.213340 23230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:51.213582 23230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:51.223152 23230 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:39687
I20260619 02:29:51.223183 23296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:39687 every 8 connection(s)
I20260619 02:29:51.224314 23230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:29:51.227707 23297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:51.234200 23297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74158b5b78fd4cd3bad1eed0d8a0c91b: Bootstrap starting.
I20260619 02:29:51.234666 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 23230
I20260619 02:29:51.234800 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:29:51.235121 17795 external_mini_cluster.cc:1428] Setting key 1103d50fbbfeef350435bc62dbebe0be
I20260619 02:29:51.237447 23297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74158b5b78fd4cd3bad1eed0d8a0c91b: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:51.238389 23297 log.cc:824] T 00000000000000000000000000000000 P 74158b5b78fd4cd3bad1eed0d8a0c91b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:51.240574 23297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74158b5b78fd4cd3bad1eed0d8a0c91b: No bootstrap required, opened a new log
I20260619 02:29:51.244959 23297 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74158b5b78fd4cd3bad1eed0d8a0c91b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74158b5b78fd4cd3bad1eed0d8a0c91b" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 39687 } }
I20260619 02:29:51.245301 23297 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74158b5b78fd4cd3bad1eed0d8a0c91b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:51.245368 23297 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74158b5b78fd4cd3bad1eed0d8a0c91b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74158b5b78fd4cd3bad1eed0d8a0c91b, State: Initialized, Role: FOLLOWER
I20260619 02:29:51.245913 23297 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74158b5b78fd4cd3bad1eed0d8a0c91b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74158b5b78fd4cd3bad1eed0d8a0c91b" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 39687 } }
I20260619 02:29:51.246057 23297 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74158b5b78fd4cd3bad1eed0d8a0c91b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:29:51.246181 23297 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74158b5b78fd4cd3bad1eed0d8a0c91b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:29:51.246306 23297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74158b5b78fd4cd3bad1eed0d8a0c91b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:51.246330 23300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:51.236696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35116 (local address 127.17.96.254:39687)
0619 02:29:51.237465 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:51.237480 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:51.237555 (+ 75us) server_negotiation.cc:407] Connection header received
0619 02:29:51.238403 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:51.238423 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:51.238943 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:51.239328 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:51.239818 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:51.240969 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:51.242041 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:51.242597 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:51.242804 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:51.243831 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:51.243837 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:51.243853 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:29:51.244334 (+ 481us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:51.244399 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:29:51.244422 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:51.244550 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:51.244762 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 02:29:51.245104 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":155,"threads_started":1}
I20260619 02:29:51.247673 23297 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74158b5b78fd4cd3bad1eed0d8a0c91b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74158b5b78fd4cd3bad1eed0d8a0c91b" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 39687 } }
I20260619 02:29:51.248075 23297 leader_election.cc:302] T 00000000000000000000000000000000 P 74158b5b78fd4cd3bad1eed0d8a0c91b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74158b5b78fd4cd3bad1eed0d8a0c91b; no voters:
I20260619 02:29:51.248489 23297 leader_election.cc:288] T 00000000000000000000000000000000 P 74158b5b78fd4cd3bad1eed0d8a0c91b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:29:51.248584 23302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74158b5b78fd4cd3bad1eed0d8a0c91b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:51.248828 23302 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74158b5b78fd4cd3bad1eed0d8a0c91b [term 1 LEADER]: Becoming Leader. State: Replica: 74158b5b78fd4cd3bad1eed0d8a0c91b, State: Running, Role: LEADER
I20260619 02:29:51.249193 23302 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74158b5b78fd4cd3bad1eed0d8a0c91b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74158b5b78fd4cd3bad1eed0d8a0c91b" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 39687 } }
I20260619 02:29:51.249884 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:29:51.250062 23297 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74158b5b78fd4cd3bad1eed0d8a0c91b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:29:51.251281 23303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74158b5b78fd4cd3bad1eed0d8a0c91b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74158b5b78fd4cd3bad1eed0d8a0c91b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74158b5b78fd4cd3bad1eed0d8a0c91b" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 39687 } } }
I20260619 02:29:51.251441 23303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74158b5b78fd4cd3bad1eed0d8a0c91b [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:51.251891 23311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:29:51.251845 23304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74158b5b78fd4cd3bad1eed0d8a0c91b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74158b5b78fd4cd3bad1eed0d8a0c91b. Latest consensus state: current_term: 1 leader_uuid: "74158b5b78fd4cd3bad1eed0d8a0c91b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74158b5b78fd4cd3bad1eed0d8a0c91b" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 39687 } } }
I20260619 02:29:51.251967 23304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74158b5b78fd4cd3bad1eed0d8a0c91b [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:51.255632 23311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:29:51.263082 23311 catalog_manager.cc:1347] Generated new cluster ID: a1caad6ad7e14954a650cb4fdb9c62e3
I20260619 02:29:51.263204 23311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:29:51.276221 23311 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:29:51.277714 23311 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:29:51.292171 23311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74158b5b78fd4cd3bad1eed0d8a0c91b: Generated new TSK 0
I20260619 02:29:51.293090 23311 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:29:51.348454 23300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:51.307885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35120 (local address 127.17.96.254:39687)
0619 02:29:51.308084 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:51.308090 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:51.308258 (+ 168us) server_negotiation.cc:407] Connection header received
0619 02:29:51.308449 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:51.308452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:51.308529 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:51.308681 (+ 152us) server_negotiation.cc:226] Negotiated authn=JWT
0619 02:29:51.308931 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:51.309931 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:51.310939 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:51.311347 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:51.311467 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:51.311893 (+ 426us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 02:29:51.347921 (+ 36028us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 02:29:51.348284 (+ 363us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35120: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":89}
W20260619 02:29:51.348553 23300 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35120: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260619 02:29:51.348657 23326 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:29:51.307619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:39687 (local address 127.0.0.1:35120)
0619 02:29:51.308149 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:51.308172 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:51.308292 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:51.308681 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:51.308683 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:51.308691 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0619 02:29:51.308812 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:51.308817 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:51.310086 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:51.310089 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:51.310794 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:51.310802 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:51.311512 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:51.311517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:51.311630 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:51.311675 (+ 45us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 02:29:51.348201 (+ 36526us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 02:29:51.348435 (+ 234us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.96.254:39687: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":164,"threads_started":1}
I20260619 02:29:51.349411 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 23230
2026-06-19T02:29:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (383 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (145 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (141 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:29:51 dist-test-slave-4qw6 krb5kdc[23338](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:29:51 dist-test-slave-4qw6 krb5kdc[23338](info): set up 2 sockets
Jun 19 02:29:51 dist-test-slave-4qw6 krb5kdc[23338](info): commencing operation
krb5kdc: starting...
W20260619 02:29:52.766861 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:29:52 dist-test-slave-4qw6 krb5kdc[23338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:29:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:29:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:29:52.905182 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:37829
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:37829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:53.027088 23351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:53.027488 23351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:53.027572 23351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:53.032140 23351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:29:53.032269 23351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:53.032306 23351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:53.032341 23351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:29:53.032402 23351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:29:53.038311 23351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:37829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:37829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:53.039613 23351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:53.040798 23351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:53.051662 23357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:53.051718 23356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:53.051807 23351 server_base.cc:1034] running on GCE node
W20260619 02:29:53.051704 23359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:53.052682 23351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:53.054184 23351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:53.055601 23351 hybrid_clock.cc:648] HybridClock initialized: now 1781836193055549 us; error 68 us; skew 500 ppm
Jun 19 02:29:53 dist-test-slave-4qw6 krb5kdc[23338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836193, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:53.062865 23351 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:29:53.064126 23351 webserver.cc:460] Webserver started at http://127.17.96.254:33573/ using document root <none> and password file <none>
I20260619 02:29:53.064810 23351 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:53.064930 23351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:53.065203 23351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:53.067699 23351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "bdd6a1d80a604c8d8432e159b91d14d4"
format_stamp: "Formatted at 2026-06-19 02:29:53 on dist-test-slave-4qw6"
server_key: "b99ba03e7fd6523c12f30af9284e3146"
server_key_iv: "0c1dc1aaf92acd662b85e88485ed6ec0"
server_key_version: "encryptionkey@0"
I20260619 02:29:53.068411 23351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "bdd6a1d80a604c8d8432e159b91d14d4"
format_stamp: "Formatted at 2026-06-19 02:29:53 on dist-test-slave-4qw6"
server_key: "b99ba03e7fd6523c12f30af9284e3146"
server_key_iv: "0c1dc1aaf92acd662b85e88485ed6ec0"
server_key_version: "encryptionkey@0"
I20260619 02:29:53.072935 23351 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 02:29:53.075893 23366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:53.077323 23351 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:29:53.077509 23351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "bdd6a1d80a604c8d8432e159b91d14d4"
format_stamp: "Formatted at 2026-06-19 02:29:53 on dist-test-slave-4qw6"
server_key: "b99ba03e7fd6523c12f30af9284e3146"
server_key_iv: "0c1dc1aaf92acd662b85e88485ed6ec0"
server_key_version: "encryptionkey@0"
I20260619 02:29:53.077661 23351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:53.094302 23351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:53.097891 23351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:53.098166 23351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:53.106637 23351 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:37829
I20260619 02:29:53.106663 23418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:37829 every 8 connection(s)
I20260619 02:29:53.107828 23351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:29:53.111080 23419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:53.113556 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 23351
I20260619 02:29:53.113675 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:29:53.114034 17795 external_mini_cluster.cc:1428] Setting key 93b18a1455fc781638d920d302641b6c
I20260619 02:29:53.118276 23419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdd6a1d80a604c8d8432e159b91d14d4: Bootstrap starting.
I20260619 02:29:53.120905 23419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdd6a1d80a604c8d8432e159b91d14d4: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:53.121886 23419 log.cc:824] T 00000000000000000000000000000000 P bdd6a1d80a604c8d8432e159b91d14d4: Log is configured to *not* fsync() on all Append() calls
Jun 19 02:29:53 dist-test-slave-4qw6 krb5kdc[23338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:53.124382 23419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdd6a1d80a604c8d8432e159b91d14d4: No bootstrap required, opened a new log
I20260619 02:29:53.128346 23419 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdd6a1d80a604c8d8432e159b91d14d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd6a1d80a604c8d8432e159b91d14d4" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 37829 } }
I20260619 02:29:53.128612 23419 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdd6a1d80a604c8d8432e159b91d14d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:53.128649 23419 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdd6a1d80a604c8d8432e159b91d14d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdd6a1d80a604c8d8432e159b91d14d4, State: Initialized, Role: FOLLOWER
I20260619 02:29:53.129201 23419 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdd6a1d80a604c8d8432e159b91d14d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd6a1d80a604c8d8432e159b91d14d4" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 37829 } }
I20260619 02:29:53.129392 23419 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdd6a1d80a604c8d8432e159b91d14d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:29:53.129551 23419 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdd6a1d80a604c8d8432e159b91d14d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:29:53.129674 23419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdd6a1d80a604c8d8432e159b91d14d4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:53.130704 23422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:53.115730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59282 (local address 127.17.96.254:37829)
0619 02:29:53.116264 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:53.116289 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:53.116342 (+ 53us) server_negotiation.cc:407] Connection header received
0619 02:29:53.117128 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:53.117148 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:53.117673 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:53.118167 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:53.119092 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:53.120209 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:53.121223 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:53.121871 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:53.122074 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:53.125153 (+ 3079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:53.125160 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:53.125185 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:53.125217 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:53.127270 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:53.127837 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:53.127843 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:53.127849 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:53.127975 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:53.128288 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:53.128293 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:53.128295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:53.128858 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:53.129090 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:53.129422 (+ 332us) server_negotiation.cc:299] Negotiation successful
0619 02:29:53.129883 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":151,"threads_started":1}
I20260619 02:29:53.130796 23419 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdd6a1d80a604c8d8432e159b91d14d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd6a1d80a604c8d8432e159b91d14d4" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 37829 } }
I20260619 02:29:53.131178 23419 leader_election.cc:302] T 00000000000000000000000000000000 P bdd6a1d80a604c8d8432e159b91d14d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdd6a1d80a604c8d8432e159b91d14d4; no voters:
I20260619 02:29:53.131532 23419 leader_election.cc:288] T 00000000000000000000000000000000 P bdd6a1d80a604c8d8432e159b91d14d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:29:53.131656 23424 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdd6a1d80a604c8d8432e159b91d14d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:53.131927 23424 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdd6a1d80a604c8d8432e159b91d14d4 [term 1 LEADER]: Becoming Leader. State: Replica: bdd6a1d80a604c8d8432e159b91d14d4, State: Running, Role: LEADER
I20260619 02:29:53.132305 23424 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdd6a1d80a604c8d8432e159b91d14d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd6a1d80a604c8d8432e159b91d14d4" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 37829 } }
I20260619 02:29:53.132826 23419 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdd6a1d80a604c8d8432e159b91d14d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:29:53.134220 23425 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdd6a1d80a604c8d8432e159b91d14d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdd6a1d80a604c8d8432e159b91d14d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd6a1d80a604c8d8432e159b91d14d4" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 37829 } } }
I20260619 02:29:53.134372 23425 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdd6a1d80a604c8d8432e159b91d14d4 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:53.134784 23433 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:29:53.135062 23426 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdd6a1d80a604c8d8432e159b91d14d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdd6a1d80a604c8d8432e159b91d14d4. Latest consensus state: current_term: 1 leader_uuid: "bdd6a1d80a604c8d8432e159b91d14d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd6a1d80a604c8d8432e159b91d14d4" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 37829 } } }
I20260619 02:29:53.135210 23426 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdd6a1d80a604c8d8432e159b91d14d4 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:53.138208 23433 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:29:53.145696 23433 catalog_manager.cc:1347] Generated new cluster ID: b2c542f688144e0ab0bd8a82dcde9a3a
I20260619 02:29:53.145923 23433 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:29:53.164157 23433 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:29:53.165313 23433 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:29:53.183045 23433 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdd6a1d80a604c8d8432e159b91d14d4: Generated new TSK 0
I20260619 02:29:53.184046 23433 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.96.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
I20260619 02:29:53.233891 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:37829
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:53.369096 23445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:53.369488 23445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:53.369544 23445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:53.374339 23445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:53.374608 23445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:53.374801 23445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:29:53.382874 23445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:37829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:53.384204 23445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:53.385319 23445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:53.396801 23453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:53.396885 23450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:53.396854 23451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:53.397320 23445 server_base.cc:1034] running on GCE node
I20260619 02:29:53.397917 23445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:53.398706 23445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:53.399919 23445 hybrid_clock.cc:648] HybridClock initialized: now 1781836193399903 us; error 37 us; skew 500 ppm
Jun 19 02:29:53 dist-test-slave-4qw6 krb5kdc[23338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836193, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:53.403458 23445 init.cc:377] Logged in from keytab as kudu/127.17.96.193@KRBTEST.COM (short username kudu)
I20260619 02:29:53.405032 23445 webserver.cc:460] Webserver started at http://127.17.96.193:36719/ using document root <none> and password file <none>
I20260619 02:29:53.405948 23445 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:53.406060 23445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:53.406311 23445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:53.409896 23445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "f955fe9f2c304785985c883921ccf282"
format_stamp: "Formatted at 2026-06-19 02:29:53 on dist-test-slave-4qw6"
server_key: "f3aacf2c8ed856fa2ecf0a3d1ff6a952"
server_key_iv: "2edb3b09f9acb991a10cccd30b963c8a"
server_key_version: "encryptionkey@0"
I20260619 02:29:53.410880 23445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "f955fe9f2c304785985c883921ccf282"
format_stamp: "Formatted at 2026-06-19 02:29:53 on dist-test-slave-4qw6"
server_key: "f3aacf2c8ed856fa2ecf0a3d1ff6a952"
server_key_iv: "2edb3b09f9acb991a10cccd30b963c8a"
server_key_version: "encryptionkey@0"
I20260619 02:29:53.416815 23445 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260619 02:29:53.420975 23460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:53.423558 23445 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 02:29:53.423805 23445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "f955fe9f2c304785985c883921ccf282"
format_stamp: "Formatted at 2026-06-19 02:29:53 on dist-test-slave-4qw6"
server_key: "f3aacf2c8ed856fa2ecf0a3d1ff6a952"
server_key_iv: "2edb3b09f9acb991a10cccd30b963c8a"
server_key_version: "encryptionkey@0"
I20260619 02:29:53.424022 23445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:53.445727 23445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:53.451241 23445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:53.451534 23445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:53.452530 23445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:53.454083 23445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:53.454167 23445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:53.454260 23445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:53.454308 23445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:53.467254 23445 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:46475
I20260619 02:29:53.467360 23573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:46475 every 8 connection(s)
I20260619 02:29:53.468402 23445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:29:53.473301 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 23445
I20260619 02:29:53.473472 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:29:53.473978 17795 external_mini_cluster.cc:1428] Setting key d980e506a4f27cd004e5201735dc8378
Jun 19 02:29:53 dist-test-slave-4qw6 krb5kdc[23338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836193, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.96.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.194@KRBTEST.COM" created.
I20260619 02:29:53.490053 23422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:53.470742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:49651 (local address 127.17.96.254:37829)
0619 02:29:53.470925 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:53.470929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:53.471954 (+ 1025us) server_negotiation.cc:407] Connection header received
0619 02:29:53.473999 (+ 2045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:53.474002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:53.474091 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:53.474195 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:53.476079 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:53.476651 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:53.478509 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:53.478945 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:53.479081 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:53.484749 (+ 5668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:53.484754 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:53.484757 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:53.484802 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:53.486987 (+ 2185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:53.487808 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:53.487811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:53.487825 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:53.487967 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:53.488492 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:53.488495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:53.488496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:53.488717 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:53.488814 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:53.489665 (+ 851us) server_negotiation.cc:299] Negotiation successful
0619 02:29:53.489865 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 02:29:53.491290 23576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:53.471175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:37829 (local address 127.17.96.193:49651)
0619 02:29:53.471750 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:53.471810 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:53.473649 (+ 1839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:53.474465 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:53.474479 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:53.475091 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:53.475894 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:53.475920 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:53.476816 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:53.476824 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:53.478353 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:53.478365 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:53.480713 (+ 2348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:53.480722 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:53.481019 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:53.481970 (+ 951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:53.482000 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:53.484375 (+ 2375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:53.487177 (+ 2802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:53.487184 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:53.487210 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:53.487641 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:53.488111 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:53.488117 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:53.488122 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:53.488340 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:53.488854 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:53.488865 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:53.489325 (+ 460us) client_negotiation.cc:769] Sending connection context
0619 02:29:53.489611 (+ 286us) client_negotiation.cc:240] Negotiation successful
0619 02:29:53.490120 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":141,"threads_started":1}
I20260619 02:29:53.492928 23574 heartbeater.cc:347] Connected to a master server at 127.17.96.254:37829
I20260619 02:29:53.493271 23574 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:53.493957 23574 heartbeater.cc:511] Master 127.17.96.254:37829 requested a full tablet report, sending...
I20260619 02:29:53.495805 23383 ts_manager.cc:194] Registered new tserver with Master: f955fe9f2c304785985c883921ccf282 (127.17.96.193:46475)
I20260619 02:29:53.497155 23383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:49651
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
I20260619 02:29:53.509245 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:37829
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:53.647516 23579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:53.647835 23579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:53.647886 23579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:53.652642 23579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:53.652736 23579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:53.652868 23579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:29:53.659132 23579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:37829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:53.660359 23579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:53.661480 23579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:53.673167 23585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:53.673084 23587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:53.673344 23579 server_base.cc:1034] running on GCE node
W20260619 02:29:53.673539 23584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:53.674125 23579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:53.674870 23579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:53.676093 23579 hybrid_clock.cc:648] HybridClock initialized: now 1781836193676075 us; error 41 us; skew 500 ppm
Jun 19 02:29:53 dist-test-slave-4qw6 krb5kdc[23338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836193, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:53.679682 23579 init.cc:377] Logged in from keytab as kudu/127.17.96.194@KRBTEST.COM (short username kudu)
I20260619 02:29:53.680716 23579 webserver.cc:460] Webserver started at http://127.17.96.194:37099/ using document root <none> and password file <none>
I20260619 02:29:53.681378 23579 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:53.681484 23579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:53.681805 23579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:53.684458 23579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "a32ea2a09b844def8dbfacb0bfe212c8"
format_stamp: "Formatted at 2026-06-19 02:29:53 on dist-test-slave-4qw6"
server_key: "df82df8f11c1dcb4b5525c7e31a4f098"
server_key_iv: "a5eaf84e46220bb13a7c5b7cfacc47f3"
server_key_version: "encryptionkey@0"
I20260619 02:29:53.685132 23579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "a32ea2a09b844def8dbfacb0bfe212c8"
format_stamp: "Formatted at 2026-06-19 02:29:53 on dist-test-slave-4qw6"
server_key: "df82df8f11c1dcb4b5525c7e31a4f098"
server_key_iv: "a5eaf84e46220bb13a7c5b7cfacc47f3"
server_key_version: "encryptionkey@0"
I20260619 02:29:53.690138 23579 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 02:29:53.693882 23594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:53.695750 23579 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260619 02:29:53.696081 23579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "a32ea2a09b844def8dbfacb0bfe212c8"
format_stamp: "Formatted at 2026-06-19 02:29:53 on dist-test-slave-4qw6"
server_key: "df82df8f11c1dcb4b5525c7e31a4f098"
server_key_iv: "a5eaf84e46220bb13a7c5b7cfacc47f3"
server_key_version: "encryptionkey@0"
I20260619 02:29:53.696374 23579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:53.719695 23579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:53.724005 23579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:53.724273 23579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:53.724975 23579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:53.726045 23579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:53.726125 23579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:53.726225 23579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:53.726279 23579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:53.737159 23579 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:45131
I20260619 02:29:53.737180 23707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:45131 every 8 connection(s)
I20260619 02:29:53.738561 23579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:29:53.738976 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 23579
I20260619 02:29:53.739120 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:29:53.739506 17795 external_mini_cluster.cc:1428] Setting key f5a8f5a53bebf69e9f7876541b8edab2
WARNING: no policy specified for kudu/127.17.96.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.195@KRBTEST.COM" created.
Jun 19 02:29:53 dist-test-slave-4qw6 krb5kdc[23338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836193, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:53.758844 23422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:53.741330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:46439 (local address 127.17.96.254:37829)
0619 02:29:53.741475 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:53.741481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:53.743115 (+ 1634us) server_negotiation.cc:407] Connection header received
0619 02:29:53.744138 (+ 1023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:53.744142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:53.744218 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:53.744337 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:53.745779 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:53.746366 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:53.747712 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:53.747960 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:53.748093 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:53.754062 (+ 5969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:53.754066 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:53.754069 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:53.754103 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:53.756045 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:53.756786 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:53.756791 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:53.756793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:53.756902 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:53.757350 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:53.757356 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:53.757358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:53.757609 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:53.757777 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:53.758442 (+ 665us) server_negotiation.cc:299] Negotiation successful
0619 02:29:53.758606 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 02:29:53.759819 23710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:53.741981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:37829 (local address 127.17.96.194:46439)
0619 02:29:53.742947 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:53.743010 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:53.743902 (+ 892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:53.744547 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:53.744556 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:53.744993 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:53.745570 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:53.745585 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:53.746521 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:53.746525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:53.747554 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:53.747566 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:53.748145 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:53.748150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:53.748357 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:53.749074 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:53.749095 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:53.753738 (+ 4643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:53.756231 (+ 2493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:53.756240 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:53.756265 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:53.756616 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:53.757023 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:53.757028 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:53.757030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:53.757225 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:53.757777 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:53.757788 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:53.758116 (+ 328us) client_negotiation.cc:769] Sending connection context
0619 02:29:53.758373 (+ 257us) client_negotiation.cc:240] Negotiation successful
0619 02:29:53.758797 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":182,"threads_started":1}
I20260619 02:29:53.761667 23708 heartbeater.cc:347] Connected to a master server at 127.17.96.254:37829
I20260619 02:29:53.762053 23708 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:53.762728 23708 heartbeater.cc:511] Master 127.17.96.254:37829 requested a full tablet report, sending...
I20260619 02:29:53.764142 23383 ts_manager.cc:194] Registered new tserver with Master: a32ea2a09b844def8dbfacb0bfe212c8 (127.17.96.194:45131)
I20260619 02:29:53.764907 23383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:46439
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
I20260619 02:29:53.770706 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:37829
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:53.906437 23713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:53.906806 23713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:53.906860 23713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:53.911880 23713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:53.911978 23713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:53.912053 23713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:29:53.919800 23713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:37829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:53.921151 23713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:53.922899 23713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:53.935254 23719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:53.935266 23721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:53.935266 23718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:53.935730 23713 server_base.cc:1034] running on GCE node
I20260619 02:29:53.936360 23713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:53.937202 23713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:53.938419 23713 hybrid_clock.cc:648] HybridClock initialized: now 1781836193938404 us; error 40 us; skew 500 ppm
Jun 19 02:29:53 dist-test-slave-4qw6 krb5kdc[23338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836193, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:53.942312 23713 init.cc:377] Logged in from keytab as kudu/127.17.96.195@KRBTEST.COM (short username kudu)
I20260619 02:29:53.943578 23713 webserver.cc:460] Webserver started at http://127.17.96.195:37541/ using document root <none> and password file <none>
I20260619 02:29:53.944299 23713 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:53.944412 23713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:53.944707 23713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:53.947291 23713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "73c0c4b0572d49b385c601ca08d9bccf"
format_stamp: "Formatted at 2026-06-19 02:29:53 on dist-test-slave-4qw6"
server_key: "00a3c3afa230c7f81846a78cae6e6887"
server_key_iv: "b77b7f1f25b19d6f718563a5f991ef3e"
server_key_version: "encryptionkey@0"
I20260619 02:29:53.948011 23713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "73c0c4b0572d49b385c601ca08d9bccf"
format_stamp: "Formatted at 2026-06-19 02:29:53 on dist-test-slave-4qw6"
server_key: "00a3c3afa230c7f81846a78cae6e6887"
server_key_iv: "b77b7f1f25b19d6f718563a5f991ef3e"
server_key_version: "encryptionkey@0"
I20260619 02:29:53.953364 23713 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260619 02:29:53.956542 23728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:53.958114 23713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:29:53.958395 23713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "73c0c4b0572d49b385c601ca08d9bccf"
format_stamp: "Formatted at 2026-06-19 02:29:53 on dist-test-slave-4qw6"
server_key: "00a3c3afa230c7f81846a78cae6e6887"
server_key_iv: "b77b7f1f25b19d6f718563a5f991ef3e"
server_key_version: "encryptionkey@0"
I20260619 02:29:53.958580 23713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:53.987350 23713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:53.991014 23713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:53.991269 23713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:53.991982 23713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:53.992985 23713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:53.993067 23713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:53.993147 23713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:53.993196 23713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:54.004061 23713 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:39917
I20260619 02:29:54.004086 23841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:39917 every 8 connection(s)
I20260619 02:29:54.005077 23713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:29:54.010452 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 23713
I20260619 02:29:54.010619 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:29:54.010953 17795 external_mini_cluster.cc:1428] Setting key 2a89e985881aedd2326c8da6844442ad
Jun 19 02:29:54 dist-test-slave-4qw6 krb5kdc[23338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836193, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:29:54.020860 23422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:54.007098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:55063 (local address 127.17.96.254:37829)
0619 02:29:54.007295 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:54.007300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:54.008191 (+ 891us) server_negotiation.cc:407] Connection header received
0619 02:29:54.009636 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:54.009639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:54.009692 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:54.009810 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:54.011617 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:54.012342 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:54.013969 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:54.014199 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:54.014300 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:54.017037 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:54.017040 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:54.017043 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:54.017069 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:54.018639 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:54.019206 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:54.019221 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:54.019227 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:54.019307 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:54.019679 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:54.019683 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:54.019685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:54.019896 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:54.020008 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:54.020547 (+ 539us) server_negotiation.cc:299] Negotiation successful
0619 02:29:54.020681 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 02:29:54.021842 23844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:54.007531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:37829 (local address 127.17.96.195:55063)
0619 02:29:54.008015 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:54.008062 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:54.009378 (+ 1316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:54.010140 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:54.010153 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:54.010755 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:54.011469 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:54.011488 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:54.012513 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:54.012518 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:54.013848 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:54.013861 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:54.014350 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:54.014362 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:54.014520 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:54.015138 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:54.015156 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:54.016824 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:54.018776 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:54.018783 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:54.018801 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:54.019090 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:54.019407 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:54.019411 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:54.019414 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:54.019565 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:54.020007 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:54.020016 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:54.020293 (+ 277us) client_negotiation.cc:769] Sending connection context
0619 02:29:54.020506 (+ 213us) client_negotiation.cc:240] Negotiation successful
0619 02:29:54.020888 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":125,"threads_started":1}
I20260619 02:29:54.023069 23842 heartbeater.cc:347] Connected to a master server at 127.17.96.254:37829
I20260619 02:29:54.023355 23842 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:54.023934 23842 heartbeater.cc:511] Master 127.17.96.254:37829 requested a full tablet report, sending...
I20260619 02:29:54.025187 23383 ts_manager.cc:194] Registered new tserver with Master: 73c0c4b0572d49b385c601ca08d9bccf (127.17.96.195:39917)
I20260619 02:29:54.025700 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:29:54.025959 23383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:55063
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/test-admin.keytab.
Jun 19 02:29:54 dist-test-slave-4qw6 krb5kdc[23338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:54.044225 17795 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260619 02:29:54.047401 23849 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:29:54.045829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55936 (local address 127.17.96.193:46475)
0619 02:29:54.046179 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:54.046186 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:54.046213 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:29:54.046291 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:54.046298 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:54.046506 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:54.046639 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:54.047180 (+ 541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55936: BlockingRecv error: recv got EOF from 127.0.0.1:55936 (error 108)
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":115,"threads_started":1}
W20260619 02:29:54.047477 17795 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/krb5cc' of type FILE
Jun 19 02:29:54 dist-test-slave-4qw6 krb5kdc[23338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:54.048777 17795 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 02:29:54 dist-test-slave-4qw6 krb5kdc[23338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:54.049981 17795 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 02:29:54 dist-test-slave-4qw6 krb5kdc[23338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.193@KRBTEST.COM
I20260619 02:29:54.060806 23849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:54.051078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55942 (local address 127.17.96.193:46475)
0619 02:29:54.051243 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:54.051247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:54.051265 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:29:54.051413 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:54.051416 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:54.051466 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:54.051549 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:54.052127 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:54.052959 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:54.054059 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:54.054476 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:54.054563 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:54.057259 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:54.057269 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:54.057286 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:54.057324 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:54.059057 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:54.059494 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:54.059499 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:54.059500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:54.059579 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:54.059818 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:54.059820 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:54.059822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:54.060076 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:54.060248 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:54.060436 (+ 188us) server_negotiation.cc:299] Negotiation successful
0619 02:29:54.060629 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260619 02:29:54.066334 23849 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:29:54.065117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55954 (local address 127.17.96.193:46475)
0619 02:29:54.065283 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:54.065286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:54.065321 (+ 35us) server_negotiation.cc:407] Connection header received
0619 02:29:54.065419 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:54.065421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:54.065487 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:54.065584 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:54.066177 (+ 593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55954: BlockingRecv error: recv got EOF from 127.0.0.1:55954 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260619 02:29:54.066792 17795 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/krb5cc)
Jun 19 02:29:54 dist-test-slave-4qw6 krb5kdc[23338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:54.067951 17795 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 02:29:54 dist-test-slave-4qw6 krb5kdc[23338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:54.068956 17795 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 02:29:54 dist-test-slave-4qw6 krb5kdc[23338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.193@KRBTEST.COM
I20260619 02:29:54.079402 23849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:54.070192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55964 (local address 127.17.96.193:46475)
0619 02:29:54.070356 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:54.070363 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:54.070511 (+ 148us) server_negotiation.cc:407] Connection header received
0619 02:29:54.070631 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:54.070634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:54.070685 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:54.070762 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:54.071297 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:54.071866 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:54.072764 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:54.073025 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:54.073125 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:54.076261 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:54.076265 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:54.076267 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:54.076299 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:54.077723 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:54.078233 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:54.078235 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:54.078237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:54.078315 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:54.078549 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:54.078551 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:54.078552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:54.078736 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:54.078858 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:54.079056 (+ 198us) server_negotiation.cc:299] Negotiation successful
0619 02:29:54.079197 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260619 02:29:54.082334 23849 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:29:54.081327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55968 (local address 127.17.96.193:46475)
0619 02:29:54.081506 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:54.081510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:54.081531 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:29:54.081649 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:54.081652 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:54.081691 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:54.081841 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:54.082192 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55968: BlockingRecv error: recv got EOF from 127.0.0.1:55968 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
W20260619 02:29:54.082432 17795 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781836162001484-17795-0/krb5kdc/krb5cc' of type FILE
Jun 19 02:29:54 dist-test-slave-4qw6 krb5kdc[23338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:54.083550 17795 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 02:29:54 dist-test-slave-4qw6 krb5kdc[23338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:54.084573 17795 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 02:29:54 dist-test-slave-4qw6 krb5kdc[23338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.193@KRBTEST.COM
I20260619 02:29:54.095803 23849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:54.085558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55970 (local address 127.17.96.193:46475)
0619 02:29:54.085725 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:54.085729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:54.085884 (+ 155us) server_negotiation.cc:407] Connection header received
0619 02:29:54.086044 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:54.086048 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:54.086098 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:54.086196 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:54.086742 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:54.087286 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:54.088133 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:54.088396 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:54.088519 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:54.091874 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:54.091879 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:54.091881 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:54.091925 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:54.093556 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:54.094205 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:54.094210 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:54.094212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:54.094307 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:54.094614 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:54.094618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:54.094620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:54.094848 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:54.095048 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:54.095290 (+ 242us) server_negotiation.cc:299] Negotiation successful
0619 02:29:54.095555 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 02:29:54.097311 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 23445
I20260619 02:29:54.104004 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 23579
I20260619 02:29:54.111632 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 23713
I20260619 02:29:54.117699 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 23351
2026-06-19T02:29:54Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2483 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:29:54 dist-test-slave-4qw6 krb5kdc[23862](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:29:54 dist-test-slave-4qw6 krb5kdc[23862](info): set up 2 sockets
Jun 19 02:29:54 dist-test-slave-4qw6 krb5kdc[23862](info): commencing operation
krb5kdc: starting...
W20260619 02:29:55.234866 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:29:55 dist-test-slave-4qw6 krb5kdc[23862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:29:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:29:55Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.254.keytab.
Entry for principal oryx/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.254.keytab.
I20260619 02:29:55.378535 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:33963
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.254.keytab
--principal=oryx/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:42083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:33963
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:55.510076 23875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:55.510428 23875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:55.510476 23875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:55.515748 23875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:29:55.515892 23875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:55.515918 23875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:55.515942 23875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:29:55.515961 23875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:29:55.525326 23875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:42083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:33963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.254.keytab
--principal=oryx/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:33963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:55.526746 23875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:55.527968 23875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:55.538085 23880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:55.538085 23881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:55.538208 23883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:55.538563 23875 server_base.cc:1034] running on GCE node
I20260619 02:29:55.539383 23875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:55.540777 23875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:55.541993 23875 hybrid_clock.cc:648] HybridClock initialized: now 1781836195541972 us; error 40 us; skew 500 ppm
Jun 19 02:29:55 dist-test-slave-4qw6 krb5kdc[23862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836195, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:55.545351 23875 init.cc:377] Logged in from keytab as oryx/127.17.96.254@KRBTEST.COM (short username oryx)
I20260619 02:29:55.546564 23875 webserver.cc:460] Webserver started at http://127.17.96.254:43425/ using document root <none> and password file <none>
I20260619 02:29:55.547305 23875 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:55.547410 23875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:55.547686 23875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:55.550354 23875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "9556c261ac4a46dc97c064bef6d74c73"
format_stamp: "Formatted at 2026-06-19 02:29:55 on dist-test-slave-4qw6"
server_key: "c78d013ee4dfaae6984cd757570f8b30"
server_key_iv: "519ff025a730d7db8c2bb786e9807c2d"
server_key_version: "encryptionkey@0"
I20260619 02:29:55.551057 23875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "9556c261ac4a46dc97c064bef6d74c73"
format_stamp: "Formatted at 2026-06-19 02:29:55 on dist-test-slave-4qw6"
server_key: "c78d013ee4dfaae6984cd757570f8b30"
server_key_iv: "519ff025a730d7db8c2bb786e9807c2d"
server_key_version: "encryptionkey@0"
I20260619 02:29:55.555667 23875 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:29:55.559165 23890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:55.561019 23875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:29:55.561237 23875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "9556c261ac4a46dc97c064bef6d74c73"
format_stamp: "Formatted at 2026-06-19 02:29:55 on dist-test-slave-4qw6"
server_key: "c78d013ee4dfaae6984cd757570f8b30"
server_key_iv: "519ff025a730d7db8c2bb786e9807c2d"
server_key_version: "encryptionkey@0"
I20260619 02:29:55.561388 23875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:55.574200 23875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:55.578133 23875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:55.578361 23875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:55.586136 23875 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:33963
I20260619 02:29:55.586447 23942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:33963 every 8 connection(s)
I20260619 02:29:55.587381 23875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:29:55.588070 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 23875
I20260619 02:29:55.588214 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:29:55.588526 17795 external_mini_cluster.cc:1428] Setting key eda72b14cef580ccb266fd7d7d25a11a
I20260619 02:29:55.591245 23943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 02:29:55 dist-test-slave-4qw6 krb5kdc[23862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM
I20260619 02:29:55.599871 23943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73: Bootstrap starting.
I20260619 02:29:55.603462 23943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:55.604519 23943 log.cc:824] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:55.606491 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:55.590238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51386 (local address 127.17.96.254:33963)
0619 02:29:55.590682 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:55.590708 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:55.590745 (+ 37us) server_negotiation.cc:407] Connection header received
0619 02:29:55.591497 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:55.591516 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:55.591901 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:55.592248 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:55.593413 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:55.594710 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:55.596076 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:55.596674 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:55.596817 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:55.600300 (+ 3483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:55.600309 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:55.600328 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:55.600377 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:55.602756 (+ 2379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:55.603332 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:55.603342 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:55.603351 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:55.603546 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:55.603987 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:55.603990 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:55.603991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:55.604492 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:55.604720 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:55.605119 (+ 399us) server_negotiation.cc:299] Negotiation successful
0619 02:29:55.605505 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":149,"threads_started":1}
I20260619 02:29:55.607321 23943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73: No bootstrap required, opened a new log
I20260619 02:29:55.612079 23943 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } }
I20260619 02:29:55.612437 23943 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:55.612540 23943 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9556c261ac4a46dc97c064bef6d74c73, State: Initialized, Role: FOLLOWER
I20260619 02:29:55.613093 23943 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } }
I20260619 02:29:55.613308 23943 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:29:55.613392 23943 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:29:55.613528 23943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:55.614964 23943 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } }
I20260619 02:29:55.615435 23943 leader_election.cc:302] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9556c261ac4a46dc97c064bef6d74c73; no voters:
I20260619 02:29:55.615871 23943 leader_election.cc:288] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:29:55.616051 23948 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:55.616353 23948 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [term 1 LEADER]: Becoming Leader. State: Replica: 9556c261ac4a46dc97c064bef6d74c73, State: Running, Role: LEADER
I20260619 02:29:55.616768 23948 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } }
I20260619 02:29:55.618090 23943 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:29:55.618925 23949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9556c261ac4a46dc97c064bef6d74c73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } }
I20260619 02:29:55.619125 23949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:55.619450 23950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9556c261ac4a46dc97c064bef6d74c73. Latest consensus state: current_term: 1 leader_uuid: "9556c261ac4a46dc97c064bef6d74c73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } }
I20260619 02:29:55.619694 23950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:55.620373 23953 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:29:55.623947 23953 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:29:55.631330 23953 catalog_manager.cc:1347] Generated new cluster ID: 6d424aa9cac949179a6e4c6bf65b8ec0
I20260619 02:29:55.631449 23953 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:29:55.670212 23953 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:29:55.671307 23953 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:29:55.689729 23953 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73: Generated new TSK 0
I20260619 02:29:55.690840 23953 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.96.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.96.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.96.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.193.keytab.
Entry for principal oryx/127.17.96.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.193.keytab.
I20260619 02:29:55.732656 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:33963
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.193.keytab
--principal=oryx/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:42083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:55.859665 23969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:55.859997 23969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:55.860046 23969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:55.864728 23969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:55.864827 23969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:55.864894 23969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:29:55.871666 23969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:42083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.193.keytab
--principal=oryx/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:33963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:55.873008 23969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:55.874359 23969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:55.885818 23977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:55.885844 23974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:55.885844 23975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:55.886119 23969 server_base.cc:1034] running on GCE node
I20260619 02:29:55.886698 23969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:55.887502 23969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:55.888677 23969 hybrid_clock.cc:648] HybridClock initialized: now 1781836195888662 us; error 36 us; skew 500 ppm
Jun 19 02:29:55 dist-test-slave-4qw6 krb5kdc[23862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836195, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:55.891965 23969 init.cc:377] Logged in from keytab as oryx/127.17.96.193@KRBTEST.COM (short username oryx)
I20260619 02:29:55.893114 23969 webserver.cc:460] Webserver started at http://127.17.96.193:45103/ using document root <none> and password file <none>
I20260619 02:29:55.893743 23969 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:55.893884 23969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:55.894466 23969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:55.897585 23969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "153c108b8f4f4fce97375af0dcae499b"
format_stamp: "Formatted at 2026-06-19 02:29:55 on dist-test-slave-4qw6"
server_key: "92bf71b96ff958e8b907faf1528d7015"
server_key_iv: "6096378caede62a508cb7c387448a55b"
server_key_version: "encryptionkey@0"
I20260619 02:29:55.898605 23969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "153c108b8f4f4fce97375af0dcae499b"
format_stamp: "Formatted at 2026-06-19 02:29:55 on dist-test-slave-4qw6"
server_key: "92bf71b96ff958e8b907faf1528d7015"
server_key_iv: "6096378caede62a508cb7c387448a55b"
server_key_version: "encryptionkey@0"
I20260619 02:29:55.904225 23969 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:29:55.907550 23984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:55.909214 23969 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:29:55.909363 23969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "153c108b8f4f4fce97375af0dcae499b"
format_stamp: "Formatted at 2026-06-19 02:29:55 on dist-test-slave-4qw6"
server_key: "92bf71b96ff958e8b907faf1528d7015"
server_key_iv: "6096378caede62a508cb7c387448a55b"
server_key_version: "encryptionkey@0"
I20260619 02:29:55.909490 23969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:55.931770 23969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:55.935670 23969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:55.935932 23969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:55.936766 23969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:55.937935 23969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:55.938009 23969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:55.938102 23969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:55.938153 23969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:55.949656 23969 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:37611
I20260619 02:29:55.949729 24097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:37611 every 8 connection(s)
I20260619 02:29:55.950819 23969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:29:55.951958 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 23969
I20260619 02:29:55.952118 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:29:55.952451 17795 external_mini_cluster.cc:1428] Setting key b8955b9345d372c2932dd0db78a75a3f
Jun 19 02:29:55 dist-test-slave-4qw6 krb5kdc[23862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836195, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.17.96.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.96.194@KRBTEST.COM" created.
I20260619 02:29:55.969581 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:55.953971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:49121 (local address 127.17.96.254:33963)
0619 02:29:55.954170 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:55.954176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:55.955450 (+ 1274us) server_negotiation.cc:407] Connection header received
0619 02:29:55.957560 (+ 2110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:55.957563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:55.957631 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:55.957734 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:55.959162 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:55.959755 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:55.961146 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:55.961426 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:55.961562 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:55.965065 (+ 3503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:55.965070 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:55.965072 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:55.965105 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:55.966978 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:55.967632 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:55.967636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:55.967640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:55.967732 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:55.968135 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:55.968138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:55.968140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:55.968388 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:55.968508 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:55.969285 (+ 777us) server_negotiation.cc:299] Negotiation successful
0619 02:29:55.969422 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 02:29:55.971009 24100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:55.954570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.193:49121)
0619 02:29:55.955214 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:55.955278 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:55.957200 (+ 1922us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:55.957995 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:55.958004 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:55.958428 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:55.958990 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:55.959006 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:55.959930 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:55.959939 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:55.960970 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:55.960981 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:55.961624 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:55.961629 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:55.961905 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:55.962727 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:55.962747 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:55.964798 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:55.967139 (+ 2341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:55.967146 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:55.967168 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:55.967489 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:55.967836 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:55.967840 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:55.967842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:55.968016 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:55.968543 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:55.968550 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:55.968937 (+ 387us) client_negotiation.cc:769] Sending connection context
0619 02:29:55.969197 (+ 260us) client_negotiation.cc:240] Negotiation successful
0619 02:29:55.969701 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":182,"threads_started":1}
I20260619 02:29:55.972538 24098 heartbeater.cc:347] Connected to a master server at 127.17.96.254:33963
I20260619 02:29:55.972852 24098 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:55.973578 24098 heartbeater.cc:511] Master 127.17.96.254:33963 requested a full tablet report, sending...
I20260619 02:29:55.975426 23907 ts_manager.cc:194] Registered new tserver with Master: 153c108b8f4f4fce97375af0dcae499b (127.17.96.193:37611)
I20260619 02:29:55.976739 23907 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:49121
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.96.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.194.keytab.
Entry for principal oryx/127.17.96.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.194.keytab.
I20260619 02:29:55.987134 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:33963
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.194.keytab
--principal=oryx/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:42083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:56.113430 24103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:56.113890 24103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:56.114007 24103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:56.119020 24103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:56.119138 24103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:56.119242 24103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:29:56.126099 24103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:42083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.194.keytab
--principal=oryx/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:33963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:56.127447 24103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:56.128669 24103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:56.140661 24109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:56.140661 24111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:56.140806 24103 server_base.cc:1034] running on GCE node
W20260619 02:29:56.140678 24108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:56.141542 24103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:56.142405 24103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:56.143608 24103 hybrid_clock.cc:648] HybridClock initialized: now 1781836196143584 us; error 41 us; skew 500 ppm
Jun 19 02:29:56 dist-test-slave-4qw6 krb5kdc[23862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836196, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:56.147194 24103 init.cc:377] Logged in from keytab as oryx/127.17.96.194@KRBTEST.COM (short username oryx)
I20260619 02:29:56.148360 24103 webserver.cc:460] Webserver started at http://127.17.96.194:44321/ using document root <none> and password file <none>
I20260619 02:29:56.149030 24103 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:56.149103 24103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:56.149403 24103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:56.151953 24103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "b84f0480fc2c41e691849d8a341d0ffe"
format_stamp: "Formatted at 2026-06-19 02:29:56 on dist-test-slave-4qw6"
server_key: "0812afde534a83235224cf461380ac57"
server_key_iv: "ab31aa2ad40f2daa6c640eff693dbea6"
server_key_version: "encryptionkey@0"
I20260619 02:29:56.152631 24103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "b84f0480fc2c41e691849d8a341d0ffe"
format_stamp: "Formatted at 2026-06-19 02:29:56 on dist-test-slave-4qw6"
server_key: "0812afde534a83235224cf461380ac57"
server_key_iv: "ab31aa2ad40f2daa6c640eff693dbea6"
server_key_version: "encryptionkey@0"
I20260619 02:29:56.156908 24103 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260619 02:29:56.159746 24118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:56.161069 24103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:29:56.161221 24103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "b84f0480fc2c41e691849d8a341d0ffe"
format_stamp: "Formatted at 2026-06-19 02:29:56 on dist-test-slave-4qw6"
server_key: "0812afde534a83235224cf461380ac57"
server_key_iv: "ab31aa2ad40f2daa6c640eff693dbea6"
server_key_version: "encryptionkey@0"
I20260619 02:29:56.161342 24103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:56.175849 24103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:56.179201 24103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:56.179426 24103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:56.180099 24103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:56.181007 24103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:56.181056 24103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:56.181133 24103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:56.181180 24103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:56.191638 24103 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:33775
I20260619 02:29:56.191728 24231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:33775 every 8 connection(s)
I20260619 02:29:56.192690 24103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:29:56.196229 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 24103
I20260619 02:29:56.196383 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:29:56.196812 17795 external_mini_cluster.cc:1428] Setting key 223885f47960a909780ee56c39aa867d
Jun 19 02:29:56 dist-test-slave-4qw6 krb5kdc[23862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836196, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.17.96.195@KRBTEST.COM; defaulting to no policy
I20260619 02:29:56.209681 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:56.194750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:47561 (local address 127.17.96.254:33963)
0619 02:29:56.194903 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:56.194907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:56.195761 (+ 854us) server_negotiation.cc:407] Connection header received
0619 02:29:56.197611 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:56.197614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:56.197676 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:56.197798 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:56.199474 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.200016 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.201366 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.201804 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.201932 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:56.204680 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:56.204688 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:56.204694 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:56.204731 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:56.206759 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:56.207378 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:56.207382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:56.207384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:56.207481 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:56.207829 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:56.207833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:56.207836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:56.208124 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:56.208219 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:56.209405 (+ 1186us) server_negotiation.cc:299] Negotiation successful
0619 02:29:56.209541 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.96.195@KRBTEST.COM" created.
I20260619 02:29:56.211894 24234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:56.195091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.194:47561)
0619 02:29:56.195590 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:56.195637 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:56.197393 (+ 1756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:56.198000 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:56.198010 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:56.198550 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:56.199303 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:56.199328 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.200158 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.200165 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:56.201224 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:56.201235 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.202005 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.202009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:56.202182 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:56.202876 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:56.202894 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:56.204464 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:56.206918 (+ 2454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:56.206923 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:56.206939 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:56.207254 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:56.207588 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:56.207590 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:56.207591 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:56.207745 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:56.208359 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:56.208370 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:56.209032 (+ 662us) client_negotiation.cc:769] Sending connection context
0619 02:29:56.209345 (+ 313us) client_negotiation.cc:240] Negotiation successful
0619 02:29:56.210087 (+ 742us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":124,"threads_started":1}
I20260619 02:29:56.213631 24232 heartbeater.cc:347] Connected to a master server at 127.17.96.254:33963
I20260619 02:29:56.214022 24232 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:56.214671 24232 heartbeater.cc:511] Master 127.17.96.254:33963 requested a full tablet report, sending...
I20260619 02:29:56.215984 23907 ts_manager.cc:194] Registered new tserver with Master: b84f0480fc2c41e691849d8a341d0ffe (127.17.96.194:33775)
I20260619 02:29:56.217023 23907 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:47561
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.96.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.195.keytab.
Entry for principal oryx/127.17.96.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.195.keytab.
I20260619 02:29:56.233018 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:33963
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.195.keytab
--principal=oryx/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:42083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:56.379276 24237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:56.379705 24237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:56.379761 24237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:56.384909 24237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:56.385032 24237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:56.385110 24237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:29:56.393102 24237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:42083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.195.keytab
--principal=oryx/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:33963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:56.394438 24237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:56.395607 24237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:56.409736 24243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:56.409742 24245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:56.409736 24242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:56.410153 24237 server_base.cc:1034] running on GCE node
I20260619 02:29:56.410686 24237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:56.411506 24237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:56.412846 24237 hybrid_clock.cc:648] HybridClock initialized: now 1781836196412825 us; error 35 us; skew 500 ppm
Jun 19 02:29:56 dist-test-slave-4qw6 krb5kdc[23862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836196, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:56.417024 24237 init.cc:377] Logged in from keytab as oryx/127.17.96.195@KRBTEST.COM (short username oryx)
I20260619 02:29:56.418699 24237 webserver.cc:460] Webserver started at http://127.17.96.195:44183/ using document root <none> and password file <none>
I20260619 02:29:56.419834 24237 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:56.419982 24237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:56.420327 24237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:56.422945 24237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "2d786c7d81db48f28b4cf8241046399f"
format_stamp: "Formatted at 2026-06-19 02:29:56 on dist-test-slave-4qw6"
server_key: "c73509d74a3a0739525414a22ec8b79f"
server_key_iv: "150d4915e97067b6154cd578b567316f"
server_key_version: "encryptionkey@0"
I20260619 02:29:56.423533 24237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "2d786c7d81db48f28b4cf8241046399f"
format_stamp: "Formatted at 2026-06-19 02:29:56 on dist-test-slave-4qw6"
server_key: "c73509d74a3a0739525414a22ec8b79f"
server_key_iv: "150d4915e97067b6154cd578b567316f"
server_key_version: "encryptionkey@0"
I20260619 02:29:56.427897 24237 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260619 02:29:56.430745 24252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:56.432438 24237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:29:56.432639 24237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "2d786c7d81db48f28b4cf8241046399f"
format_stamp: "Formatted at 2026-06-19 02:29:56 on dist-test-slave-4qw6"
server_key: "c73509d74a3a0739525414a22ec8b79f"
server_key_iv: "150d4915e97067b6154cd578b567316f"
server_key_version: "encryptionkey@0"
I20260619 02:29:56.432783 24237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:56.454083 24237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:56.458292 24237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:56.458592 24237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:56.459378 24237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:56.460448 24237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:56.460538 24237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:56.460619 24237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:56.460668 24237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:56.471474 24237 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:36321
I20260619 02:29:56.471654 24365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:36321 every 8 connection(s)
I20260619 02:29:56.472560 24237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:29:56.473037 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 24237
I20260619 02:29:56.473184 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:29:56.473560 17795 external_mini_cluster.cc:1428] Setting key ed1f23fd60102d13787e3e8804e29db5
Jun 19 02:29:56 dist-test-slave-4qw6 krb5kdc[23862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836196, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM
I20260619 02:29:56.489295 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:56.474909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:40309 (local address 127.17.96.254:33963)
0619 02:29:56.475084 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:56.475089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:56.476688 (+ 1599us) server_negotiation.cc:407] Connection header received
0619 02:29:56.477686 (+ 998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:56.477688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:56.477739 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:56.477896 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:56.479325 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.479910 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.481201 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.481429 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.481524 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:56.484886 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:56.484889 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:56.484892 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:56.484922 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:56.486425 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:56.487208 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:56.487211 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:56.487212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:56.487335 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:56.487831 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:56.487834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:56.487835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:56.488128 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:56.488225 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:56.489009 (+ 784us) server_negotiation.cc:299] Negotiation successful
0619 02:29:56.489143 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 02:29:56.490480 24368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:56.475495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.195:40309)
0619 02:29:56.476529 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:56.476580 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:56.477476 (+ 896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:56.478101 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:56.478109 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:56.478564 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:56.479137 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:56.479155 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.480050 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.480053 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:56.481067 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:56.481078 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.481598 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.481601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:56.482064 (+ 463us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:56.482876 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:56.482898 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:56.484660 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:56.486605 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:56.486613 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:56.486635 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:56.487090 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:56.487458 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:56.487461 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:56.487462 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:56.487718 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:56.488258 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:56.488267 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:56.488672 (+ 405us) client_negotiation.cc:769] Sending connection context
0619 02:29:56.488965 (+ 293us) client_negotiation.cc:240] Negotiation successful
0619 02:29:56.489333 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"thread_start_us":181,"threads_started":1}
I20260619 02:29:56.491865 24366 heartbeater.cc:347] Connected to a master server at 127.17.96.254:33963
I20260619 02:29:56.492139 24366 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:56.492722 24366 heartbeater.cc:511] Master 127.17.96.254:33963 requested a full tablet report, sending...
I20260619 02:29:56.494297 23907 ts_manager.cc:194] Registered new tserver with Master: 2d786c7d81db48f28b4cf8241046399f (127.17.96.195:36321)
I20260619 02:29:56.495111 23907 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:40309
I20260619 02:29:56.501048 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.96.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.96.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.96.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.253.keytab.
Entry for principal oryx/127.17.96.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.253.keytab.
I20260619 02:29:56.535332 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.253:32859
--webserver_interface=127.17.96.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.253.keytab
--principal=oryx/127.17.96.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:42083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:33963,127.17.96.253:32859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:29:56.675091 24371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:56.675441 24371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:56.675491 24371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:56.680192 24371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:29:56.680301 24371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:56.680327 24371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:56.680349 24371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:29:56.680369 24371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:29:56.686861 24371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:42083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:33963,127.17.96.253:32859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.253.keytab
--principal=oryx/127.17.96.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.253:32859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.96.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
W20260619 02:29:56.687191 24371 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.17.96.254:33963,127.17.96.253:32859'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260619 02:29:56.688221 24371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:56.689440 24371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:56.699759 24377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:56.699759 24376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:56.699788 24379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:56.700220 24371 server_base.cc:1034] running on GCE node
I20260619 02:29:56.700883 24371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:56.702373 24371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:56.703611 24371 hybrid_clock.cc:648] HybridClock initialized: now 1781836196703600 us; error 42 us; skew 500 ppm
Jun 19 02:29:56 dist-test-slave-4qw6 krb5kdc[23862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836196, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:56.707120 24371 init.cc:377] Logged in from keytab as oryx/127.17.96.253@KRBTEST.COM (short username oryx)
I20260619 02:29:56.708298 24371 webserver.cc:460] Webserver started at http://127.17.96.253:37929/ using document root <none> and password file <none>
I20260619 02:29:56.708971 24371 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:56.709079 24371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:56.709363 24371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:56.711838 24371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/data/instance:
uuid: "549beb78d8c6431184a1e0282c52e6f0"
format_stamp: "Formatted at 2026-06-19 02:29:56 on dist-test-slave-4qw6"
server_key: "3a995dae8a82c745d5f095f576f0d6e6"
server_key_iv: "aaf6a062e5cea2a7abd12bd25940b69d"
server_key_version: "encryptionkey@0"
I20260619 02:29:56.712523 24371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/wal/instance:
uuid: "549beb78d8c6431184a1e0282c52e6f0"
format_stamp: "Formatted at 2026-06-19 02:29:56 on dist-test-slave-4qw6"
server_key: "3a995dae8a82c745d5f095f576f0d6e6"
server_key_iv: "aaf6a062e5cea2a7abd12bd25940b69d"
server_key_version: "encryptionkey@0"
I20260619 02:29:56.717676 24371 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 02:29:56.721460 24386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:56.723450 24371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:29:56.723651 24371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/wal
uuid: "549beb78d8c6431184a1e0282c52e6f0"
format_stamp: "Formatted at 2026-06-19 02:29:56 on dist-test-slave-4qw6"
server_key: "3a995dae8a82c745d5f095f576f0d6e6"
server_key_iv: "aaf6a062e5cea2a7abd12bd25940b69d"
server_key_version: "encryptionkey@0"
I20260619 02:29:56.723839 24371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:56.740694 24371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:56.745483 24371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:56.745826 24371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:56.755041 24371 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.253:32859
I20260619 02:29:56.755398 24438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.253:32859 every 8 connection(s)
I20260619 02:29:56.756366 24371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/data/info.pb
I20260619 02:29:56.760609 24439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:56.765066 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 24371
I20260619 02:29:56.767068 24439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:29:56 dist-test-slave-4qw6 krb5kdc[23862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836196, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.253@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM
Jun 19 02:29:56 dist-test-slave-4qw6 krb5kdc[23862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.96.253@KRBTEST.COM
I20260619 02:29:56.781836 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:56.769210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51396 (local address 127.17.96.254:33963)
0619 02:29:56.769353 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:56.769361 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:56.769887 (+ 526us) server_negotiation.cc:407] Connection header received
0619 02:29:56.770030 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:56.770033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:56.770078 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:56.770150 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:56.770945 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.771770 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.773413 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.773832 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.773934 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:56.777028 (+ 3094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:56.777038 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:56.777041 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:56.777072 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:56.779220 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:56.779946 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:56.779950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:56.779952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:56.780058 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:56.780426 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:56.780431 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:56.780433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:56.780714 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:56.780838 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:56.781442 (+ 604us) server_negotiation.cc:299] Negotiation successful
0619 02:29:56.781600 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 02:29:56.783063 24443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:56.769077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.0.0.1:51396)
0619 02:29:56.769684 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:56.769781 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:56.769931 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:56.770249 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:56.770258 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:56.770631 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:56.770795 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:56.770804 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.771934 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.771938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:56.773289 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:56.773299 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.773992 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.773996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:56.774168 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:56.774731 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:56.774757 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:56.776821 (+ 2064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:56.779363 (+ 2542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:56.779372 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:56.779384 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:56.779818 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:56.780148 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:56.780153 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:56.780156 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:56.780315 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:56.780847 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:56.780857 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:56.781171 (+ 314us) client_negotiation.cc:769] Sending connection context
0619 02:29:56.781394 (+ 223us) client_negotiation.cc:240] Negotiation successful
0619 02:29:56.781794 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":59,"thread_start_us":130,"threads_started":1}
I20260619 02:29:56.783061 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:56.767060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35590 (local address 127.17.96.253:32859)
0619 02:29:56.767545 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:56.767618 (+ 73us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:56.767669 (+ 51us) server_negotiation.cc:407] Connection header received
0619 02:29:56.768391 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:56.768409 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:56.768868 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:56.769271 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:56.770307 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.771700 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.773013 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.773637 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.773845 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:56.777708 (+ 3863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:56.777717 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:56.777735 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:56.777794 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:56.779962 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:56.780509 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:56.780516 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:56.780518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:56.780624 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:56.780978 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:56.780984 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:56.780986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:56.781282 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:56.781547 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:56.781854 (+ 307us) server_negotiation.cc:299] Negotiation successful
0619 02:29:56.782521 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":329,"thread_start_us":172,"threads_started":1}
I20260619 02:29:56.786993 24439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:29:56 dist-test-slave-4qw6 krb5kdc[23862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836196, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.253@KRBTEST.COM for oryx/127.17.96.253@KRBTEST.COM
I20260619 02:29:56.797641 24443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:56.787735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:32859 (local address 127.0.0.1:35602)
0619 02:29:56.787910 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:56.787928 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:56.788025 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:56.788278 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:56.788280 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:56.788502 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:56.788646 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:56.788653 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.790010 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.790013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:56.790742 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:56.790752 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.791992 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.791995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:56.792108 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:56.792754 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:56.792772 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:56.794375 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:56.796013 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:56.796019 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:56.796023 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:56.796238 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:56.796536 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:56.796540 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:56.796546 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:56.796635 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:56.797058 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:56.797062 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:56.797156 (+ 94us) client_negotiation.cc:769] Sending connection context
0619 02:29:56.797267 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 02:29:56.797442 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 02:29:56.797641 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:56.787948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35602 (local address 127.17.96.253:32859)
0619 02:29:56.788063 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:56.788071 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:56.788087 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:29:56.788125 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:56.788128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:56.788166 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:56.788267 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:56.788792 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.789887 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.790870 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.791851 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.791992 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:56.794544 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:56.794548 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:56.794551 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:56.794582 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:56.795890 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:56.796350 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:56.796353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:56.796355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:56.796437 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:56.796733 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:56.796738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:56.796740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:56.796941 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:56.797057 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:56.797324 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 02:29:56.797502 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":33}
I20260619 02:29:56.801003 24439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0: Bootstrap starting.
I20260619 02:29:56.803717 24439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:56.804569 24439 log.cc:824] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:56.806766 24439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0: No bootstrap required, opened a new log
I20260619 02:29:56.810030 24439 raft_consensus.cc:348] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } }
I20260619 02:29:56.810274 24439 raft_consensus.cc:374] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:56.810336 24439 raft_consensus.cc:729] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 549beb78d8c6431184a1e0282c52e6f0, State: Initialized, Role: FOLLOWER
I20260619 02:29:56.810807 24439 consensus_queue.cc:260] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } }
I20260619 02:29:56.811354 24445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } } }
I20260619 02:29:56.811511 24445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:29:56.812170 24439 sys_catalog.cc:564] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 02:29:56.817793 24456 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:29:56.817912 24456 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:29:56.819798 24371 master_runner.cc:428] Detected that this master 549beb78d8c6431184a1e0282c52e6f0 is joining an existing cluster
I20260619 02:29:56.819869 24371 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.96.253:32859
I20260619 02:29:56.832741 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:56.822563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51408 (local address 127.17.96.254:33963)
0619 02:29:56.822757 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:56.822763 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:56.823071 (+ 308us) server_negotiation.cc:407] Connection header received
0619 02:29:56.823429 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:56.823432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:56.823495 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:56.823596 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:56.824527 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.825106 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.826441 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.826821 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.826953 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:56.828709 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:56.828713 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:56.828716 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:56.828749 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:56.830681 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:56.831302 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:56.831306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:56.831308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:56.831412 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:56.831745 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:56.831749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:56.831751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:56.831974 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:56.832103 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:56.832418 (+ 315us) server_negotiation.cc:299] Negotiation successful
0619 02:29:56.832569 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 02:29:56.832762 24462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:56.822564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.0.0.1:51408)
0619 02:29:56.822956 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:56.822977 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:56.823307 (+ 330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:56.823742 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:56.823745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:56.824086 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:56.824357 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:56.824367 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.825240 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.825246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:56.826296 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:56.826307 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.827098 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.827102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:56.827215 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:56.827899 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:56.827917 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:56.828527 (+ 610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:56.830820 (+ 2293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:56.830827 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:56.830829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:56.831136 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:56.831523 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:56.831529 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:56.831531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:56.831632 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:56.832089 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:56.832093 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:56.832235 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 02:29:56.832358 (+ 123us) client_negotiation.cc:240] Negotiation successful
0619 02:29:56.832540 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":70,"spinlock_wait_cycles":1280,"thread_start_us":161,"threads_started":1}
I20260619 02:29:56.833187 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:56.823191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35612 (local address 127.17.96.253:32859)
0619 02:29:56.823322 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:56.823327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:56.823595 (+ 268us) server_negotiation.cc:407] Connection header received
0619 02:29:56.823687 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:56.823690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:56.823736 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:56.823823 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:56.824503 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.825381 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.826568 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.826810 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.826934 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:56.828905 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:56.828912 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:56.828915 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:56.828952 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:56.831103 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:56.831631 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:56.831635 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:56.831637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:56.831727 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:56.832022 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:56.832025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:56.832027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:56.832265 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:56.832338 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:56.832634 (+ 296us) server_negotiation.cc:299] Negotiation successful
0619 02:29:56.832758 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":60}
I20260619 02:29:56.833819 24463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:56.823102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:32859 (local address 127.0.0.1:35612)
0619 02:29:56.823483 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:56.823497 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:56.823605 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:56.823852 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:56.823854 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:56.824108 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:56.824357 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:56.824367 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.825503 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.825506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:56.826449 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:56.826462 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.826945 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.826949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:56.827050 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:56.828196 (+ 1146us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:56.828209 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:56.828700 (+ 491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:56.831222 (+ 2522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:56.831226 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:56.831229 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:56.831456 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:56.831834 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:56.831837 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:56.831839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:56.831921 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:56.832397 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:56.832400 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:56.832477 (+ 77us) client_negotiation.cc:769] Sending connection context
0619 02:29:56.833457 (+ 980us) client_negotiation.cc:240] Negotiation successful
0619 02:29:56.833605 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":23,"spinlock_wait_cycles":1792,"thread_start_us":117,"threads_started":1}
Jun 19 02:29:56 dist-test-slave-4qw6 krb5kdc[23862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836195, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.254@KRBTEST.COM for oryx/127.17.96.253@KRBTEST.COM
WARNING: no policy specified for oryx/127.17.96.252@KRBTEST.COM; defaulting to no policy
I20260619 02:29:56.850966 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:56.839599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35620 (local address 127.17.96.253:32859)
0619 02:29:56.839753 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:56.839758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:56.839771 (+ 13us) server_negotiation.cc:407] Connection header received
0619 02:29:56.839836 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:56.839839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:56.839891 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:56.839983 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:56.841013 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.841865 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.842834 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.843214 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.843333 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:56.846622 (+ 3289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:56.846627 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:56.846631 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:56.846665 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:56.848700 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:56.849445 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:56.849449 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:56.849451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:56.849546 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:56.849915 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:56.849918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:56.849920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:56.850146 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:56.850241 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:56.850702 (+ 461us) server_negotiation.cc:299] Negotiation successful
0619 02:29:56.850825 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 02:29:56.851106 24464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:56.838739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:32859 (local address 127.0.0.1:35620)
0619 02:29:56.839205 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:56.839268 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:56.839405 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:56.840068 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:56.840077 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:56.840663 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:56.840868 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:56.840880 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.841990 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.841997 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:56.842707 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:56.842718 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.843357 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.843364 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:56.843522 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:56.844250 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:56.844272 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:56.846421 (+ 2149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:56.848878 (+ 2457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:56.848892 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:56.848900 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:56.849305 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:56.849666 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:56.849670 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:56.849672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:56.849800 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:56.850279 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:56.850288 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:56.850452 (+ 164us) client_negotiation.cc:769] Sending connection context
0619 02:29:56.850637 (+ 185us) client_negotiation.cc:240] Negotiation successful
0619 02:29:56.850892 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":115,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.96.252@KRBTEST.COM" created.
I20260619 02:29:56.852957 23907 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.96.253:32859: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0"
member_type: NON_VOTER
last_known_addr {
host: "127.17.96.253"
port: 32859
}
attrs {
promote: true
}
}
dest_uuid: "9556c261ac4a46dc97c064bef6d74c73"
cas_config_opid_index: -1
I20260619 02:29:56.853886 23907 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: NON_VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: true } }
I20260619 02:29:56.856004 24468 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 549beb78d8c6431184a1e0282c52e6f0 (127.17.96.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: NON_VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: true } } }
I20260619 02:29:56.856853 24413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:56.857892 24468 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.96.253:32859
I20260619 02:29:56.858398 24371 master.cc:537] Master@127.17.96.253:32859 shutting down...
I20260619 02:29:56.858428 24468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9556c261ac4a46dc97c064bef6d74c73" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: NON_VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: true } } }
I20260619 02:29:56.858536 24468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:56.858662 24413 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 1 FOLLOWER]: Refusing update from remote peer 9556c261ac4a46dc97c064bef6d74c73: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260619 02:29:56.859470 24469 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: NON_VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 02:29:56.861572 23894 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260619 02:29:56.862656 23894 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 -> Peer 549beb78d8c6431184a1e0282c52e6f0 (127.17.96.253:32859): Couldn't send request to peer 549beb78d8c6431184a1e0282c52e6f0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 02:29:56.863983 24371 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:29:56.864323 24371 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:56.864463 24371 tablet_replica.cc:330] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.96.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.252.keytab.
Entry for principal oryx/127.17.96.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.252.keytab.
I20260619 02:29:56.875041 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.252:42201
--webserver_interface=127.17.96.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.252.keytab
--principal=oryx/127.17.96.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:42083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:33963,127.17.96.253:32859,127.17.96.252:42201
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:29:56.881143 24371 master.cc:552] Master@127.17.96.253:32859 shutdown complete.
I20260619 02:29:56.881238 24371 master_runner.cc:305] Clearing existing system tablet
I20260619 02:29:56.881579 24371 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:56.881659 24371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:56.883658 24371 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:29:56.884891 24474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:56.885228 24371 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:56.885417 24371 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:29:56.885473 24371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/wal
uuid: "549beb78d8c6431184a1e0282c52e6f0"
format_stamp: "Formatted at 2026-06-19 02:29:56 on dist-test-slave-4qw6"
server_key: "3a995dae8a82c745d5f095f576f0d6e6"
server_key_iv: "aaf6a062e5cea2a7abd12bd25940b69d"
server_key_version: "encryptionkey@0"
I20260619 02:29:56.888666 24371 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:56.891587 24371 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 02:29:56.891719 24371 log.cc:1196] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260619 02:29:56.892009 24371 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0: Deleting consensus metadata
I20260619 02:29:56.892187 24371 master_runner.cc:315] Copying system tablet from 127.17.96.254:33963
I20260619 02:29:56.893596 24371 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0: tablet copy: Beginning tablet copy session from remote peer at address 127.17.96.254:33963
I20260619 02:29:56.904531 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:56.894464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51410 (local address 127.17.96.254:33963)
0619 02:29:56.894638 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:56.894643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:56.894822 (+ 179us) server_negotiation.cc:407] Connection header received
0619 02:29:56.895019 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:56.895024 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:56.895082 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:56.895140 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:56.895867 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.896933 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.898375 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.898794 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.898922 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:56.900457 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:56.900462 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:56.900465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:56.900499 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:56.902470 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:56.903052 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:56.903056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:56.903059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:56.903170 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:56.903496 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:56.903500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:56.903502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:56.903764 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:56.903878 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:56.904211 (+ 333us) server_negotiation.cc:299] Negotiation successful
0619 02:29:56.904352 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260619 02:29:56.904547 24480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:56.894313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.0.0.1:51410)
0619 02:29:56.894702 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:56.894719 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:56.894857 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:56.895301 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:56.895305 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:56.895562 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:56.895708 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:56.895716 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.897122 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.897129 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:56.898207 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:56.898220 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:56.898946 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:56.898950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:56.899066 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:56.899750 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:56.899766 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:56.900271 (+ 505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:56.902616 (+ 2345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:56.902620 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:56.902622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:56.902919 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:56.903281 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:56.903284 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:56.903286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:56.903389 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:56.903876 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:56.903880 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:56.904008 (+ 128us) client_negotiation.cc:769] Sending connection context
0619 02:29:56.904129 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 02:29:56.904307 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":121,"threads_started":1}
I20260619 02:29:56.905690 23927 tablet_copy_service.cc:140] P 9556c261ac4a46dc97c064bef6d74c73: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 549beb78d8c6431184a1e0282c52e6f0 ({username='oryx', principal='oryx/127.17.96.253@KRBTEST.COM'} at 127.0.0.1:51410)
I20260619 02:29:56.905941 23927 tablet_copy_service.cc:161] P 9556c261ac4a46dc97c064bef6d74c73: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 549beb78d8c6431184a1e0282c52e6f0 at {username='oryx', principal='oryx/127.17.96.253@KRBTEST.COM'} at 127.0.0.1:51410: session id = 549beb78d8c6431184a1e0282c52e6f0-00000000000000000000000000000000
I20260619 02:29:56.907888 23927 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 02:29:56.909832 24371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 02:29:56.912681 24371 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0: tablet copy: Starting download of 0 data blocks...
I20260619 02:29:56.912911 24371 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0: tablet copy: Starting download of 1 WAL segments...
I20260619 02:29:56.915410 24371 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 02:29:56.917673 23927 tablet_copy_service.cc:342] P 9556c261ac4a46dc97c064bef6d74c73: Request end of tablet copy session 549beb78d8c6431184a1e0282c52e6f0-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.96.253@KRBTEST.COM'} at 127.0.0.1:51410
I20260619 02:29:56.917881 23927 tablet_copy_service.cc:433] P 9556c261ac4a46dc97c064bef6d74c73: ending tablet copy session 549beb78d8c6431184a1e0282c52e6f0-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 549beb78d8c6431184a1e0282c52e6f0
I20260619 02:29:56.929407 24371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:56.931521 24481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:56.931803 24371 server_base.cc:1034] running on GCE node
W20260619 02:29:56.931857 24484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:56.933931 24482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:56.934249 24371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:56.934654 24371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:56.935817 24371 hybrid_clock.cc:648] HybridClock initialized: now 1781836196935790 us; error 50 us; skew 500 ppm
Jun 19 02:29:56 dist-test-slave-4qw6 krb5kdc[23862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836196, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:56.937820 24371 init.cc:377] Logged in from keytab as oryx/127.17.96.253@KRBTEST.COM (short username oryx)
I20260619 02:29:56.938400 24371 webserver.cc:460] Webserver started at http://127.17.96.253:40667/ using document root <none> and password file <none>
I20260619 02:29:56.938630 24371 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:56.938681 24371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:56.940290 24371 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:29:56.941636 24491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:56.942034 24371 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:29:56.942131 24371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/wal
uuid: "549beb78d8c6431184a1e0282c52e6f0"
format_stamp: "Formatted at 2026-06-19 02:29:56 on dist-test-slave-4qw6"
server_key: "3a995dae8a82c745d5f095f576f0d6e6"
server_key_iv: "aaf6a062e5cea2a7abd12bd25940b69d"
server_key_version: "encryptionkey@0"
I20260619 02:29:56.942231 24371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:56.974768 24371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:56.977480 24371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:56.979596 24098 heartbeater.cc:503] Master 127.17.96.254:33963 was elected leader, sending a full tablet report...
I20260619 02:29:56.983845 24371 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.253:32859
I20260619 02:29:56.983883 24543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.253:32859 every 8 connection(s)
I20260619 02:29:56.984649 24371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-1/data/info.pb
I20260619 02:29:56.987004 24544 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:29:56.988299 24544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0: Bootstrap starting.
I20260619 02:29:57.001947 24544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:29:57.002326 24544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0: Bootstrap complete.
I20260619 02:29:57.003069 24544 raft_consensus.cc:348] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: NON_VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: true } }
I20260619 02:29:57.003232 24544 raft_consensus.cc:729] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 549beb78d8c6431184a1e0282c52e6f0, State: Initialized, Role: LEARNER
I20260619 02:29:57.003358 24544 consensus_queue.cc:260] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: NON_VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: true } }
I20260619 02:29:57.003896 24544 sys_catalog.cc:564] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:29:57.003840 24547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: NON_VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: true } } }
I20260619 02:29:57.004134 24547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [sys.catalog]: This master's current role is: LEARNER
I20260619 02:29:57.007575 24558 catalog_manager.cc:1259] Loaded cluster ID: 6d424aa9cac949179a6e4c6bf65b8ec0
I20260619 02:29:57.007652 24558 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0: loading cluster ID for follower catalog manager: success
I20260619 02:29:57.009989 24558 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0: acquiring CA information for follower catalog manager: success
I20260619 02:29:57.011698 24558 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 02:29:57.034868 24472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:57.035266 24472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:57.035372 24472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:57.040369 24472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:29:57.040489 24472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:57.040529 24472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:57.040592 24472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:29:57.040647 24472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:29:57.047178 24472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:42083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:33963,127.17.96.253:32859,127.17.96.252:42201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.252.keytab
--principal=oryx/127.17.96.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.252:42201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.96.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:57.048565 24472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:57.049857 24472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:57.060906 24566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:57.060920 24563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:57.060925 24564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:57.060990 24472 server_base.cc:1034] running on GCE node
I20260619 02:29:57.062230 24472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:57.063668 24472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:57.064900 24472 hybrid_clock.cc:648] HybridClock initialized: now 1781836197064808 us; error 150 us; skew 500 ppm
Jun 19 02:29:57 dist-test-slave-4qw6 krb5kdc[23862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836197, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:57.069041 24472 init.cc:377] Logged in from keytab as oryx/127.17.96.252@KRBTEST.COM (short username oryx)
I20260619 02:29:57.070320 24472 webserver.cc:460] Webserver started at http://127.17.96.252:45393/ using document root <none> and password file <none>
I20260619 02:29:57.071022 24472 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:57.071131 24472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:57.071404 24472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:29:57.074142 24472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/data/instance:
uuid: "34f02c2e89484735991408738f401ee8"
format_stamp: "Formatted at 2026-06-19 02:29:57 on dist-test-slave-4qw6"
server_key: "c7bd251eeded2e2216cd559806da79a3"
server_key_iv: "0b3d8bd7251089daeea45c3a0d510a0f"
server_key_version: "encryptionkey@0"
I20260619 02:29:57.074853 24472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/wal/instance:
uuid: "34f02c2e89484735991408738f401ee8"
format_stamp: "Formatted at 2026-06-19 02:29:57 on dist-test-slave-4qw6"
server_key: "c7bd251eeded2e2216cd559806da79a3"
server_key_iv: "0b3d8bd7251089daeea45c3a0d510a0f"
server_key_version: "encryptionkey@0"
I20260619 02:29:57.079510 24472 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 02:29:57.082573 24573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:57.084163 24472 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:29:57.084353 24472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/wal
uuid: "34f02c2e89484735991408738f401ee8"
format_stamp: "Formatted at 2026-06-19 02:29:57 on dist-test-slave-4qw6"
server_key: "c7bd251eeded2e2216cd559806da79a3"
server_key_iv: "0b3d8bd7251089daeea45c3a0d510a0f"
server_key_version: "encryptionkey@0"
I20260619 02:29:57.084503 24472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:57.103164 24472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:57.107388 24472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:57.107658 24472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:57.116293 24472 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.252:42201
I20260619 02:29:57.116644 24625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.252:42201 every 8 connection(s)
I20260619 02:29:57.117439 24472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/data/info.pb
I20260619 02:29:57.121450 24626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:57.124063 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 24472
I20260619 02:29:57.127332 24626 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:29:57 dist-test-slave-4qw6 krb5kdc[23862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.96.252@KRBTEST.COM
Jun 19 02:29:57 dist-test-slave-4qw6 krb5kdc[23862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836197, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.252@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM
I20260619 02:29:57.142550 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.129849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51418 (local address 127.17.96.254:33963)
0619 02:29:57.129989 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:57.129994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:57.130261 (+ 267us) server_negotiation.cc:407] Connection header received
0619 02:29:57.130435 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:57.130438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:57.130496 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:57.130573 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:57.131560 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.132121 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.133204 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.133452 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.133588 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.137008 (+ 3420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:57.137013 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:57.137016 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:57.137049 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:57.139640 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.140446 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.140450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.140452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.140572 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.141017 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.141021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.141026 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.141306 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:57.141440 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:57.142195 (+ 755us) server_negotiation.cc:299] Negotiation successful
0619 02:29:57.142353 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 02:29:57.143621 24629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.129569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.0.0.1:51418)
0619 02:29:57.130090 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:57.130149 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:57.130312 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:57.130666 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:57.130672 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:57.131182 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:57.131387 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.131400 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.132287 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.132292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.133030 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.133041 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.133635 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.133640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.133840 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.134451 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:57.134470 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:57.136741 (+ 2271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:57.139808 (+ 3067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.139819 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.139830 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.140240 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.140707 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.140714 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.140719 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.140891 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.141570 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:57.141576 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:57.141873 (+ 297us) client_negotiation.cc:769] Sending connection context
0619 02:29:57.142099 (+ 226us) client_negotiation.cc:240] Negotiation successful
0619 02:29:57.142617 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":113,"thread_start_us":202,"threads_started":1}
I20260619 02:29:57.143622 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.126085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41856 (local address 127.17.96.252:42201)
0619 02:29:57.126663 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:57.126682 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:57.126726 (+ 44us) server_negotiation.cc:407] Connection header received
0619 02:29:57.127633 (+ 907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:57.127654 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:57.128075 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:57.128482 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:57.129484 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.131267 (+ 1783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.132249 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.132820 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.133025 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.136851 (+ 3826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:57.136861 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:57.136880 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:57.136921 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:57.139838 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.140457 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.140464 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.140466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.140623 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.140980 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.140987 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.140990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.141601 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:57.141887 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:57.142215 (+ 328us) server_negotiation.cc:299] Negotiation successful
0619 02:29:57.142466 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":199,"threads_started":1}
I20260619 02:29:57.145011 24626 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:29:57 dist-test-slave-4qw6 krb5kdc[23862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836197, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.252@KRBTEST.COM for oryx/127.17.96.253@KRBTEST.COM
I20260619 02:29:57.155081 24629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.145598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:32859 (local address 127.0.0.1:35636)
0619 02:29:57.145732 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:57.145774 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:57.145865 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:57.146364 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:57.146368 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:57.146581 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:57.146690 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.146696 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.147690 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.147697 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.148290 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.148299 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.148883 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.148892 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.149006 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.149723 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:57.149736 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:57.151334 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:57.153312 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.153316 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.153318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.153547 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.153935 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.153939 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.153941 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.154032 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.154478 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:57.154481 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:57.154594 (+ 113us) client_negotiation.cc:769] Sending connection context
0619 02:29:57.154711 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 02:29:57.154880 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260619 02:29:57.155093 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.145767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35636 (local address 127.17.96.253:32859)
0619 02:29:57.146109 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:57.146114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:57.146129 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:29:57.146177 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:57.146181 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:57.146243 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:57.146351 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:57.146823 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.147568 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.148426 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.148761 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.148885 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.151515 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:57.151527 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:57.151530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:57.151558 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:57.153163 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.153664 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.153669 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.153671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.153819 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.154135 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.154141 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.154143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.154365 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:57.154488 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:57.154779 (+ 291us) server_negotiation.cc:299] Negotiation successful
0619 02:29:57.154920 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":105,"threads_started":1}
I20260619 02:29:57.156155 24626 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:29:57 dist-test-slave-4qw6 krb5kdc[23862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836197, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.252@KRBTEST.COM for oryx/127.17.96.252@KRBTEST.COM
I20260619 02:29:57.167174 24629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.156743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:42201 (local address 127.0.0.1:41858)
0619 02:29:57.156856 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:57.156869 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:57.156936 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:57.157202 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:57.157204 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:57.157345 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:57.157426 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.157431 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.159198 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.159201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.159745 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.159754 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.160573 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.160577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.160660 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.161235 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:57.161248 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:57.162943 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:57.165137 (+ 2194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.165141 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.165143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.165356 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.165654 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.165656 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.165658 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.165905 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.166412 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:57.166419 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:57.166543 (+ 124us) client_negotiation.cc:769] Sending connection context
0619 02:29:57.166730 (+ 187us) client_negotiation.cc:240] Negotiation successful
0619 02:29:57.166897 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 02:29:57.167325 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.156862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41858 (local address 127.17.96.252:42201)
0619 02:29:57.157007 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:57.157011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:57.157024 (+ 13us) server_negotiation.cc:407] Connection header received
0619 02:29:57.157058 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:57.157060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:57.157100 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:57.157180 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:57.157568 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.158295 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.160156 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.160406 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.160517 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.163147 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:57.163152 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:57.163155 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:57.163181 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:57.165008 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.165469 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.165473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.165474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.165562 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.165999 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.166002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.166003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.166269 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:57.166415 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:57.166827 (+ 412us) server_negotiation.cc:299] Negotiation successful
0619 02:29:57.167015 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 02:29:57.171397 24626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8: Bootstrap starting.
I20260619 02:29:57.174275 24626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:57.175134 24626 log.cc:824] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:57.177443 24626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8: No bootstrap required, opened a new log
I20260619 02:29:57.180823 24626 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } }
I20260619 02:29:57.181080 24626 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:57.181115 24626 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34f02c2e89484735991408738f401ee8, State: Initialized, Role: FOLLOWER
I20260619 02:29:57.181676 24626 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } }
I20260619 02:29:57.182543 24632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } } }
I20260619 02:29:57.182825 24632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:29:57.183247 24626 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 02:29:57.190610 24643 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:29:57.190879 24643 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:29:57.195194 24472 master_runner.cc:428] Detected that this master 34f02c2e89484735991408738f401ee8 is joining an existing cluster
I20260619 02:29:57.195314 24472 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.96.252:42201
I20260619 02:29:57.197821 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 23969
I20260619 02:29:57.206337 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:37611
--local_ip_for_outbound_sockets=127.17.96.193
--tserver_master_addrs=127.17.96.254:33963
--webserver_port=45103
--webserver_interface=127.17.96.193
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.193.keytab
--principal=oryx/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:42083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.96.254:33963,127.17.96.253:32859
--tserver_master_addrs=127.17.96.254:33963,127.17.96.253:32859,127.17.96.252:42201 with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:29:57.210376 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.199837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41874 (local address 127.17.96.252:42201)
0619 02:29:57.199998 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:57.200006 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:57.200027 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:29:57.200083 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:57.200086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:57.200147 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:57.200239 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:57.201055 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.202052 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.203393 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.203779 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.203861 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.206403 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:57.206408 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:57.206411 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:57.206436 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:57.208397 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.208952 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.208956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.208958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.209037 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.209350 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.209353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.209355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.209586 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:57.209663 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:57.210027 (+ 364us) server_negotiation.cc:299] Negotiation successful
0619 02:29:57.210169 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":57}
I20260619 02:29:57.210966 24649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.199272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:42201 (local address 127.0.0.1:41874)
0619 02:29:57.199708 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:57.199730 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:57.199871 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:57.200398 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:57.200401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:57.200729 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:57.200925 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.200934 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.202207 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.202214 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.203245 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.203258 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.204043 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.204047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.204315 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.205641 (+ 1326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:57.205659 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:57.206235 (+ 576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:57.208555 (+ 2320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.208559 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.208561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.208842 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.209165 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.209168 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.209171 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.209253 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.209729 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:57.209732 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:57.209847 (+ 115us) client_negotiation.cc:769] Sending connection context
0619 02:29:57.210629 (+ 782us) client_negotiation.cc:240] Negotiation successful
0619 02:29:57.210779 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":52,"thread_start_us":130,"threads_started":1}
I20260619 02:29:57.219969 24232 heartbeater.cc:503] Master 127.17.96.254:33963 was elected leader, sending a full tablet report...
I20260619 02:29:57.222414 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.200762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35638 (local address 127.17.96.253:32859)
0619 02:29:57.200906 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:57.200910 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:57.208209 (+ 7299us) server_negotiation.cc:407] Connection header received
0619 02:29:57.208437 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:57.208444 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:57.208535 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:57.208602 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:57.210893 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.211937 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.213971 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.214382 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.214524 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.216304 (+ 1780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:57.216308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:57.216311 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:57.216343 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:57.218406 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.219634 (+ 1228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.219641 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.219644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.219756 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.220292 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.220296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.220298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.220552 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:57.221625 (+ 1073us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:57.222063 (+ 438us) server_negotiation.cc:299] Negotiation successful
0619 02:29:57.222225 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 02:29:57.223017 24651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.200230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.0.0.1:51430)
0619 02:29:57.201146 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:57.201162 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:57.201312 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:57.201685 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:57.201688 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:57.202018 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:57.202164 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.202171 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.203349 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.203352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.204297 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.204308 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.205082 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.205089 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.205309 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.218346 (+ 13037us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:57.218369 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:57.218917 (+ 548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:57.221196 (+ 2279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.221199 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.221202 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.221488 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.221885 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.221889 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.221891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.221979 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.222436 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:57.222439 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:57.222556 (+ 117us) client_negotiation.cc:769] Sending connection context
0619 02:29:57.222657 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 02:29:57.222815 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"mutex_wait_us":7,"thread_start_us":107,"threads_started":1}
I20260619 02:29:57.223532 24650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.200722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:32859 (local address 127.0.0.1:35638)
0619 02:29:57.208081 (+ 7359us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:57.208103 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:57.208304 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:57.208803 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:57.208809 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:57.209092 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:57.209275 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.209285 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.212486 (+ 3201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.212491 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.213808 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.213826 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.214570 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.214578 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.214721 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.215409 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:57.215431 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:57.216094 (+ 663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:57.219141 (+ 3047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.219146 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.219152 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.219445 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.219899 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.219905 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.219909 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.220172 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.221690 (+ 1518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:57.221694 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:57.221847 (+ 153us) client_negotiation.cc:769] Sending connection context
0619 02:29:57.222907 (+ 1060us) client_negotiation.cc:240] Negotiation successful
0619 02:29:57.223311 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7247,"thread_start_us":82,"threads_started":1}
I20260619 02:29:57.223554 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.200292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51430 (local address 127.17.96.254:33963)
0619 02:29:57.200449 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:57.200454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:57.201405 (+ 951us) server_negotiation.cc:407] Connection header received
0619 02:29:57.201476 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:57.201480 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:57.201538 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:57.201632 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:57.202320 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.203188 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.204453 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.204805 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.204887 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.219134 (+ 14247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:57.219139 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:57.219142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:57.219184 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:57.221062 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.221624 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.221627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.221629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.221715 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.222082 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.222085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.222087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.222306 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:57.222394 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:57.223242 (+ 848us) server_negotiation.cc:299] Negotiation successful
0619 02:29:57.223370 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
Jun 19 02:29:57 dist-test-slave-4qw6 krb5kdc[23862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836195, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.254@KRBTEST.COM for oryx/127.17.96.252@KRBTEST.COM
I20260619 02:29:57.239802 24464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.228624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:42201 (local address 127.0.0.1:41882)
0619 02:29:57.228903 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:57.228919 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:57.229035 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:57.229658 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:57.229662 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:57.229942 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:57.230072 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.230080 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.231288 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.231293 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.232250 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.232261 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.232921 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.232925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.233034 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.233656 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:57.233673 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:57.235468 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:57.237880 (+ 2412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.237884 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.237886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.238190 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.238560 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.238563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.238565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.238672 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.239164 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:57.239168 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:57.239294 (+ 126us) client_negotiation.cc:769] Sending connection context
0619 02:29:57.239410 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 02:29:57.239610 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192}
I20260619 02:29:57.239904 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.229205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41882 (local address 127.17.96.252:42201)
0619 02:29:57.229346 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:57.229350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:57.229363 (+ 13us) server_negotiation.cc:407] Connection header received
0619 02:29:57.229438 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:57.229441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:57.229506 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:57.229600 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:57.230214 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.231161 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.232394 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.232772 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.232899 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.235654 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:57.235658 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:57.235661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:57.235694 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:57.237653 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.238322 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.238326 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.238328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.238432 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.238789 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.238792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.238794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.239050 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:57.239156 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:57.239516 (+ 360us) server_negotiation.cc:299] Negotiation successful
0619 02:29:57.239656 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 02:29:57.241858 23907 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.96.252:42201: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "34f02c2e89484735991408738f401ee8"
member_type: NON_VOTER
last_known_addr {
host: "127.17.96.252"
port: 42201
}
attrs {
promote: true
}
}
dest_uuid: "9556c261ac4a46dc97c064bef6d74c73"
cas_config_opid_index: 5
I20260619 02:29:57.242619 23907 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: NON_VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: true } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: NON_VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: true } }
I20260619 02:29:57.243839 24469 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 34f02c2e89484735991408738f401ee8 (127.17.96.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: NON_VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: true } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: NON_VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: true } } }
I20260619 02:29:57.245245 24600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:57.245644 24469 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.96.252:42201
I20260619 02:29:57.245884 24469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9556c261ac4a46dc97c064bef6d74c73" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: NON_VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: true } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: NON_VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: true } } }
I20260619 02:29:57.245993 24469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:57.247438 24600 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 1 FOLLOWER]: Refusing update from remote peer 9556c261ac4a46dc97c064bef6d74c73: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 02:29:57.248196 24468 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: NON_VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:29:57.253218 24464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.244060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:32859 (local address 127.0.0.1:35640)
0619 02:29:57.248139 (+ 4079us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:57.248157 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:57.248268 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:57.248729 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:57.248732 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:57.248751 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:57.248854 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.248861 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.250092 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.250096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.251456 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.251474 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.252717 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.252721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.252821 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.252873 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 02:29:57.252952 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 02:29:57.253055 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3993}
I20260619 02:29:57.253696 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.244322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35640 (local address 127.17.96.253:32859)
0619 02:29:57.248022 (+ 3700us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:57.248028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:57.248361 (+ 333us) server_negotiation.cc:407] Connection header received
0619 02:29:57.248423 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:57.248426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:57.248494 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:57.248587 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:57.249005 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.249925 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.251636 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.252369 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.252484 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.252642 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:57.253487 (+ 845us) server_negotiation.cc:299] Negotiation successful
0619 02:29:57.253578 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3573}
I20260619 02:29:57.254251 24518 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 1 LEARNER]: Refusing update from remote peer 9556c261ac4a46dc97c064bef6d74c73: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 02:29:57.254585 24600 raft_consensus.cc:858] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: NON_VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: NON_VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: true } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: NON_VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: true } }
I20260619 02:29:57.254765 24468 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: NON_VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260619 02:29:57.254967 24600 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: NON_VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: NON_VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: true } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: NON_VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: true } }
I20260619 02:29:57.255259 24472 master.cc:537] Master@127.17.96.252:42201 shutting down...
I20260619 02:29:57.256848 24518 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 34f02c2e89484735991408738f401ee8 (127.17.96.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: NON_VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: true } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: NON_VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: true } } }
I20260619 02:29:57.257421 24600 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 549beb78d8c6431184a1e0282c52e6f0 (127.17.96.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: NON_VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: true } } }
I20260619 02:29:57.259948 24632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9556c261ac4a46dc97c064bef6d74c73. Latest consensus state: current_term: 1 leader_uuid: "9556c261ac4a46dc97c064bef6d74c73" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: NON_VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: true } } }
I20260619 02:29:57.260069 24632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260619 02:29:57.260160 24632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9556c261ac4a46dc97c064bef6d74c73" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: NON_VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: true } } }
I20260619 02:29:57.260217 24632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260619 02:29:57.260274 24632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9556c261ac4a46dc97c064bef6d74c73" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: NON_VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: true } } }
I20260619 02:29:57.260337 24632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260619 02:29:57.261016 24547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9556c261ac4a46dc97c064bef6d74c73. Latest consensus state: current_term: 1 leader_uuid: "9556c261ac4a46dc97c064bef6d74c73" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: NON_VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: true } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: NON_VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: true } } }
I20260619 02:29:57.261164 24547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [sys.catalog]: This master's current role is: LEARNER
I20260619 02:29:57.261268 24547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9556c261ac4a46dc97c064bef6d74c73" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: NON_VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: true } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: NON_VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: true } } }
I20260619 02:29:57.261348 24547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [sys.catalog]: This master's current role is: LEARNER
I20260619 02:29:57.263942 24468 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73: attempting to promote NON_VOTER 549beb78d8c6431184a1e0282c52e6f0 to VOTER
I20260619 02:29:57.264438 24468 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: NON_VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: true } }
I20260619 02:29:57.264771 24653 mvcc.cc:205] Tried to move back new op lower bound from 7298401057536884736 to 7298401057247559680. Current Snapshot: MvccSnapshot[applied={T|T < 7298401057291583488}]
I20260619 02:29:57.266923 24472 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260619 02:29:57.267210 24472 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260619 02:29:57.267293 24472 tablet_replica.cc:330] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8: stopping tablet replica
I20260619 02:29:57.269222 24518 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 1 LEARNER]: Refusing update from remote peer 9556c261ac4a46dc97c064bef6d74c73: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260619 02:29:57.269356 23891 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 -> Peer 34f02c2e89484735991408738f401ee8 (127.17.96.252:42201): Couldn't send request to peer 34f02c2e89484735991408738f401ee8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 02:29:57.269915 24468 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260619 02:29:57.272310 24469 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 549beb78d8c6431184a1e0282c52e6f0 (127.17.96.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: NON_VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: true } } }
I20260619 02:29:57.274775 24469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9556c261ac4a46dc97c064bef6d74c73" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: NON_VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: true } } }
I20260619 02:29:57.275303 24518 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 549beb78d8c6431184a1e0282c52e6f0 (127.17.96.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: NON_VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: true } } }
I20260619 02:29:57.278023 24658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9556c261ac4a46dc97c064bef6d74c73" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: NON_VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: true } } }
I20260619 02:29:57.278174 24658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:29:57.278326 24469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:57.277606 24468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9556c261ac4a46dc97c064bef6d74c73" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: NON_VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: true } } }
I20260619 02:29:57.278477 24468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:57.283434 24472 master.cc:552] Master@127.17.96.252:42201 shutdown complete.
I20260619 02:29:57.283509 24472 master_runner.cc:305] Clearing existing system tablet
I20260619 02:29:57.283771 24472 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:57.283830 24472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:57.285328 24472 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 02:29:57.286402 24664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:57.286710 24472 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:57.286845 24472 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:29:57.286881 24472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/wal
uuid: "34f02c2e89484735991408738f401ee8"
format_stamp: "Formatted at 2026-06-19 02:29:57 on dist-test-slave-4qw6"
server_key: "c7bd251eeded2e2216cd559806da79a3"
server_key_iv: "0b3d8bd7251089daeea45c3a0d510a0f"
server_key_version: "encryptionkey@0"
I20260619 02:29:57.288967 24472 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:57.291803 24472 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 02:29:57.291906 24472 log.cc:1196] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260619 02:29:57.292212 24472 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8: Deleting consensus metadata
I20260619 02:29:57.292373 24472 master_runner.cc:315] Copying system tablet from 127.17.96.254:33963
I20260619 02:29:57.293601 24472 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8: tablet copy: Beginning tablet copy session from remote peer at address 127.17.96.254:33963
I20260619 02:29:57.305478 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.294598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51444 (local address 127.17.96.254:33963)
0619 02:29:57.294747 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:57.294752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:57.295389 (+ 637us) server_negotiation.cc:407] Connection header received
0619 02:29:57.296210 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:57.296214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:57.296274 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:57.296402 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:57.297083 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.298018 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.299305 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.299654 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.299770 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.301313 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:57.301317 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:57.301320 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:57.301350 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:57.303275 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.303879 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.303883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.303885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.303985 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.304342 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.304345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.304347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.304595 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:57.304700 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:57.305110 (+ 410us) server_negotiation.cc:299] Negotiation successful
0619 02:29:57.305248 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:29:57.306090 24670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.294557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.0.0.1:51444)
0619 02:29:57.295228 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:57.295247 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:57.295746 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:57.296446 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:57.296451 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:57.296773 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:57.296924 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.296934 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.298153 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.298157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.299152 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.299165 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.299800 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.299804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.299921 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.300602 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:57.300622 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:57.301128 (+ 506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:57.303448 (+ 2320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.303452 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.303455 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.303754 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.304105 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.304109 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.304111 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.304220 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.304719 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:57.304724 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:57.304884 (+ 160us) client_negotiation.cc:769] Sending connection context
0619 02:29:57.305020 (+ 136us) client_negotiation.cc:240] Negotiation successful
0619 02:29:57.305305 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":146,"threads_started":1}
I20260619 02:29:57.306871 23927 tablet_copy_service.cc:140] P 9556c261ac4a46dc97c064bef6d74c73: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 34f02c2e89484735991408738f401ee8 ({username='oryx', principal='oryx/127.17.96.252@KRBTEST.COM'} at 127.0.0.1:51444)
I20260619 02:29:57.307000 23927 tablet_copy_service.cc:161] P 9556c261ac4a46dc97c064bef6d74c73: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 34f02c2e89484735991408738f401ee8 at {username='oryx', principal='oryx/127.17.96.252@KRBTEST.COM'} at 127.0.0.1:51444: session id = 34f02c2e89484735991408738f401ee8-00000000000000000000000000000000
I20260619 02:29:57.308240 23927 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 02:29:57.309886 24472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 02:29:57.312793 24472 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8: tablet copy: Starting download of 0 data blocks...
I20260619 02:29:57.313159 24472 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8: tablet copy: Starting download of 1 WAL segments...
I20260619 02:29:57.315409 24472 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 02:29:57.317917 23927 tablet_copy_service.cc:342] P 9556c261ac4a46dc97c064bef6d74c73: Request end of tablet copy session 34f02c2e89484735991408738f401ee8-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.96.252@KRBTEST.COM'} at 127.0.0.1:51444
I20260619 02:29:57.318051 23927 tablet_copy_service.cc:433] P 9556c261ac4a46dc97c064bef6d74c73: ending tablet copy session 34f02c2e89484735991408738f401ee8-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 34f02c2e89484735991408738f401ee8
I20260619 02:29:57.328444 24472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:57.331051 24671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:57.331288 24472 server_base.cc:1034] running on GCE node
W20260619 02:29:57.334424 24674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:57.335574 24672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:57.335937 24472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:57.336326 24472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:57.337495 24472 hybrid_clock.cc:648] HybridClock initialized: now 1781836197337466 us; error 53 us; skew 500 ppm
Jun 19 02:29:57 dist-test-slave-4qw6 krb5kdc[23862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836197, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:57.339672 24472 init.cc:377] Logged in from keytab as oryx/127.17.96.252@KRBTEST.COM (short username oryx)
I20260619 02:29:57.340394 24472 webserver.cc:460] Webserver started at http://127.17.96.252:34653/ using document root <none> and password file <none>
I20260619 02:29:57.340646 24472 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:57.340704 24472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:57.342454 24472 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 02:29:57.343787 24681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:57.344102 24472 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:29:57.344183 24472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/wal
uuid: "34f02c2e89484735991408738f401ee8"
format_stamp: "Formatted at 2026-06-19 02:29:57 on dist-test-slave-4qw6"
server_key: "c7bd251eeded2e2216cd559806da79a3"
server_key_iv: "0b3d8bd7251089daeea45c3a0d510a0f"
server_key_version: "encryptionkey@0"
I20260619 02:29:57.344280 24472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:57.366879 24472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:57.369567 24472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:57.375157 24472 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.252:42201
I20260619 02:29:57.375313 24736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.252:42201 every 8 connection(s)
I20260619 02:29:57.375815 24472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/master-2/data/info.pb
W20260619 02:29:57.377341 24652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:57.377709 24652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:57.377820 24652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 02:29:57.378238 24737 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:29:57.379618 24737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8: Bootstrap starting.
W20260619 02:29:57.383038 24652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:57.383147 24652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:57.383268 24652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:29:57.389912 24652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:42083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.193.keytab
--principal=oryx/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:37611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=45103
--tserver_master_addrs=127.17.96.254:33963,127.17.96.253:32859,127.17.96.252:42201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:57.391132 24737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:29:57.391285 24652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:57.391546 24737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8: Bootstrap complete.
I20260619 02:29:57.392314 24737 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: NON_VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: true } }
I20260619 02:29:57.392541 24737 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34f02c2e89484735991408738f401ee8, State: Initialized, Role: LEARNER
I20260619 02:29:57.392511 24652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:29:57.392685 24737 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: NON_VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: true } }
I20260619 02:29:57.393155 24740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: NON_VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: true } } }
I20260619 02:29:57.393302 24737 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:29:57.393396 24740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [sys.catalog]: This master's current role is: LEARNER
I20260619 02:29:57.398631 24751 catalog_manager.cc:1259] Loaded cluster ID: 6d424aa9cac949179a6e4c6bf65b8ec0
I20260619 02:29:57.398718 24751 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8: loading cluster ID for follower catalog manager: success
I20260619 02:29:57.401484 24751 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8: acquiring CA information for follower catalog manager: success
I20260619 02:29:57.403307 24751 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 02:29:57.408030 24756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:57.407995 24753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:57.408068 24754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:57.408624 24652 server_base.cc:1034] running on GCE node
I20260619 02:29:57.409255 24652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:57.410144 24652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:57.411362 24652 hybrid_clock.cc:648] HybridClock initialized: now 1781836197411331 us; error 56 us; skew 500 ppm
Jun 19 02:29:57 dist-test-slave-4qw6 krb5kdc[23862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836197, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:57.415525 24652 init.cc:377] Logged in from keytab as oryx/127.17.96.193@KRBTEST.COM (short username oryx)
I20260619 02:29:57.416746 24652 webserver.cc:460] Webserver started at http://127.17.96.193:45103/ using document root <none> and password file <none>
I20260619 02:29:57.417733 24652 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:57.418009 24652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:57.424026 24652 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:29:57.427013 24763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:57.428807 24652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:29:57.429008 24652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "153c108b8f4f4fce97375af0dcae499b"
format_stamp: "Formatted at 2026-06-19 02:29:55 on dist-test-slave-4qw6"
server_key: "92bf71b96ff958e8b907faf1528d7015"
server_key_iv: "6096378caede62a508cb7c387448a55b"
server_key_version: "encryptionkey@0"
I20260619 02:29:57.429744 24652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:57.451754 24652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:57.455623 24652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:57.455874 24652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:57.456588 24652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:57.457690 24652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:57.457818 24652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:57.457948 24652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:57.457996 24652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:57.469492 24652 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:37611
I20260619 02:29:57.469525 24876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:37611 every 8 connection(s)
I20260619 02:29:57.470758 24652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:29:57.478597 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 24652
I20260619 02:29:57.478832 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 24103
Jun 19 02:29:57 dist-test-slave-4qw6 krb5kdc[23862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836197, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.252@KRBTEST.COM
Jun 19 02:29:57 dist-test-slave-4qw6 krb5kdc[23862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836197, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.253@KRBTEST.COM
I20260619 02:29:57.487699 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:33775
--local_ip_for_outbound_sockets=127.17.96.194
--tserver_master_addrs=127.17.96.254:33963
--webserver_port=44321
--webserver_interface=127.17.96.194
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.194.keytab
--principal=oryx/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:42083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.96.254:33963,127.17.96.253:32859
--tserver_master_addrs=127.17.96.254:33963,127.17.96.253:32859,127.17.96.252:42201 with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:29:57 dist-test-slave-4qw6 krb5kdc[23862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836197, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM
I20260619 02:29:57.492830 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.473439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:48629 (local address 127.17.96.252:42201)
0619 02:29:57.473940 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:57.473946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:57.474873 (+ 927us) server_negotiation.cc:407] Connection header received
0619 02:29:57.476255 (+ 1382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:57.476259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:57.476325 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:57.476412 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:57.479432 (+ 3020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.480279 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.481952 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.482353 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.482455 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.486217 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:57.486222 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:57.486225 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:57.486263 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:57.488708 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.489546 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.489552 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.489555 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.489670 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.491283 (+ 1613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.491286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.491287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.491509 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:57.491642 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:57.492446 (+ 804us) server_negotiation.cc:299] Negotiation successful
0619 02:29:57.492618 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":91,"threads_started":1}
I20260619 02:29:57.493280 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.475191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:45705 (local address 127.17.96.253:32859)
0619 02:29:57.475365 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:57.475370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:57.475612 (+ 242us) server_negotiation.cc:407] Connection header received
0619 02:29:57.476238 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:57.476242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:57.476309 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:57.476370 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:57.478403 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.479223 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.481893 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.482296 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.482437 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.486441 (+ 4004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:57.486449 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:57.486455 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:57.486501 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:57.489496 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.490261 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.490266 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.490269 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.490489 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.491071 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.491077 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.491080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.491409 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:57.491523 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:57.492464 (+ 941us) server_negotiation.cc:299] Negotiation successful
0619 02:29:57.492734 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 02:29:57.495611 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.474796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:32859 (local address 127.17.96.193:45705)
0619 02:29:57.475492 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:57.475510 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:57.475954 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:57.476843 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:57.476854 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:57.477484 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:57.478190 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.478201 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.479787 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.479795 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.481670 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.481693 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.482522 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.482530 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.482836 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.483729 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:57.483770 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:57.486233 (+ 2463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:57.489657 (+ 3424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.489661 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.489663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.489973 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.490685 (+ 712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.490688 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.490690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.490872 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.491607 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:57.491616 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:57.492107 (+ 491us) client_negotiation.cc:769] Sending connection context
0619 02:29:57.493922 (+ 1815us) client_negotiation.cc:240] Negotiation successful
0619 02:29:57.494116 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"mutex_wait_us":190,"spinlock_wait_cycles":48768,"thread_start_us":94,"threads_started":1}
I20260619 02:29:57.495607 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.473463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:42201 (local address 127.17.96.193:48629)
0619 02:29:57.474739 (+ 1276us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:57.474757 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:57.475960 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:57.476846 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:57.476855 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:57.477484 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:57.478177 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.478196 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.480441 (+ 2245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.480448 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.481694 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.481713 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.482603 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.482609 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.482818 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.483774 (+ 956us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:57.483790 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:57.485922 (+ 2132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:57.488871 (+ 2949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.488882 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.488909 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.489395 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.489836 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.489841 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.489845 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.490079 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.491632 (+ 1553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:57.491635 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:57.492106 (+ 471us) client_negotiation.cc:769] Sending connection context
0619 02:29:57.493125 (+ 1019us) client_negotiation.cc:240] Negotiation successful
0619 02:29:57.493732 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1169,"mutex_wait_us":189,"spinlock_wait_cycles":47488,"thread_start_us":224,"threads_started":1}
I20260619 02:29:57.497455 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.472971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:50771 (local address 127.17.96.254:33963)
0619 02:29:57.473154 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:57.473159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:57.474911 (+ 1752us) server_negotiation.cc:407] Connection header received
0619 02:29:57.476238 (+ 1327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:57.476242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:57.476308 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:57.476370 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:57.487251 (+ 10881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.488265 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.489675 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.490114 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.490229 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.493123 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:57.493127 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:57.493129 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:57.493163 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:57.495124 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.495733 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.495738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.495744 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.495843 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.496192 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.496196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.496199 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.496477 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:57.496576 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:57.497158 (+ 582us) server_negotiation.cc:299] Negotiation successful
0619 02:29:57.497299 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 02:29:57.497855 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.473467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.193:50771)
0619 02:29:57.474633 (+ 1166us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:57.474696 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:57.475954 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:57.476843 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:57.476854 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:57.477484 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:57.478190 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.478201 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.488463 (+ 10262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.488467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.489510 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.489523 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.490463 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.490467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.490604 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.491230 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:57.491249 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:57.492930 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:57.495289 (+ 2359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.495292 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.495294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.495589 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.495952 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.495954 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.495956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.496074 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.496847 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:57.496852 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:57.496955 (+ 103us) client_negotiation.cc:769] Sending connection context
0619 02:29:57.497089 (+ 134us) client_negotiation.cc:240] Negotiation successful
0619 02:29:57.497262 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":938,"mutex_wait_us":114,"spinlock_wait_cycles":23168,"thread_start_us":759,"threads_started":1}
I20260619 02:29:57.498046 24366 heartbeater.cc:503] Master 127.17.96.254:33963 was elected leader, sending a full tablet report...
I20260619 02:29:57.500134 24877 heartbeater.cc:347] Connected to a master server at 127.17.96.252:42201
I20260619 02:29:57.500129 24879 heartbeater.cc:347] Connected to a master server at 127.17.96.253:32859
I20260619 02:29:57.500129 24878 heartbeater.cc:347] Connected to a master server at 127.17.96.254:33963
I20260619 02:29:57.500463 24878 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:57.500465 24877 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:57.500468 24879 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:57.501230 24878 heartbeater.cc:511] Master 127.17.96.254:33963 requested a full tablet report, sending...
I20260619 02:29:57.501448 24879 heartbeater.cc:511] Master 127.17.96.253:32859 requested a full tablet report, sending...
I20260619 02:29:57.501230 24877 heartbeater.cc:511] Master 127.17.96.252:42201 requested a full tablet report, sending...
I20260619 02:29:57.502898 23907 ts_manager.cc:194] Re-registered known tserver with Master: 153c108b8f4f4fce97375af0dcae499b (127.17.96.193:37611)
I20260619 02:29:57.503221 24693 ts_manager.cc:194] Registered new tserver with Master: 153c108b8f4f4fce97375af0dcae499b (127.17.96.193:37611)
I20260619 02:29:57.503305 24499 ts_manager.cc:194] Registered new tserver with Master: 153c108b8f4f4fce97375af0dcae499b (127.17.96.193:37611)
I20260619 02:29:57.503754 23907 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:50771
W20260619 02:29:57.624248 24885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:57.624752 24885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:57.624874 24885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:57.629652 24885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:57.629839 24885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:57.629971 24885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:29:57.636332 24885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:42083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.194.keytab
--principal=oryx/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:33775
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=44321
--tserver_master_addrs=127.17.96.254:33963,127.17.96.253:32859,127.17.96.252:42201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:57.637801 24885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:57.639107 24885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:57.651588 24891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:57.651650 24885 server_base.cc:1034] running on GCE node
W20260619 02:29:57.651613 24893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:57.651660 24890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:57.652633 24885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:57.653465 24885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:57.654685 24885 hybrid_clock.cc:648] HybridClock initialized: now 1781836197654691 us; error 59 us; skew 500 ppm
Jun 19 02:29:57 dist-test-slave-4qw6 krb5kdc[23862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836197, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:57.658402 24885 init.cc:377] Logged in from keytab as oryx/127.17.96.194@KRBTEST.COM (short username oryx)
I20260619 02:29:57.659607 24885 webserver.cc:460] Webserver started at http://127.17.96.194:44321/ using document root <none> and password file <none>
I20260619 02:29:57.660288 24885 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:57.660431 24885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:57.665091 24885 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:29:57.667670 24900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:57.668992 24885 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:29:57.669179 24885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "b84f0480fc2c41e691849d8a341d0ffe"
format_stamp: "Formatted at 2026-06-19 02:29:56 on dist-test-slave-4qw6"
server_key: "0812afde534a83235224cf461380ac57"
server_key_iv: "ab31aa2ad40f2daa6c640eff693dbea6"
server_key_version: "encryptionkey@0"
I20260619 02:29:57.669854 24885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:57.688961 24885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:57.693542 24885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:57.693838 24885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:57.694679 24885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:57.695791 24885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:57.695881 24885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:57.695958 24885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:57.696017 24885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:57.701792 24464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.696409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:42201 (local address 127.0.0.1:41892)
0619 02:29:57.696630 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:57.696649 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:57.696757 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:57.697114 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:57.697119 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:57.697149 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:57.697300 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.697307 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.698307 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.698312 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.699794 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.699815 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.701044 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.701051 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.701198 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.701254 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 02:29:57.701379 (+ 125us) client_negotiation.cc:240] Negotiation successful
0619 02:29:57.701518 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129}
I20260619 02:29:57.701870 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.696551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41892 (local address 127.17.96.252:42201)
0619 02:29:57.696707 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:57.696712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:57.696732 (+ 20us) server_negotiation.cc:407] Connection header received
0619 02:29:57.696895 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:57.696898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:57.696959 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:57.697054 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:57.697435 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.698130 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.700017 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.700806 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.700984 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.701185 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:57.701459 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 02:29:57.701610 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:29:57.702993 24707 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260619 02:29:57.703572 24740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9556c261ac4a46dc97c064bef6d74c73. Latest consensus state: current_term: 1 leader_uuid: "9556c261ac4a46dc97c064bef6d74c73" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: NON_VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: true } } }
I20260619 02:29:57.703753 24740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [sys.catalog]: This master's current role is: LEARNER
I20260619 02:29:57.709021 24885 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:33775
I20260619 02:29:57.709051 25013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:33775 every 8 connection(s)
I20260619 02:29:57.710305 24885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:29:57.718974 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 24885
I20260619 02:29:57.719187 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 24237
Jun 19 02:29:57 dist-test-slave-4qw6 krb5kdc[23862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836197, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.252@KRBTEST.COM
Jun 19 02:29:57 dist-test-slave-4qw6 krb5kdc[23862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836197, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM
Jun 19 02:29:57 dist-test-slave-4qw6 krb5kdc[23862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836197, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.253@KRBTEST.COM
I20260619 02:29:57.730445 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:36321
--local_ip_for_outbound_sockets=127.17.96.195
--tserver_master_addrs=127.17.96.254:33963
--webserver_port=44183
--webserver_interface=127.17.96.195
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.195.keytab
--principal=oryx/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:42083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.96.254:33963,127.17.96.253:32859
--tserver_master_addrs=127.17.96.254:33963,127.17.96.253:32859,127.17.96.252:42201 with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:29:57.732867 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.713563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:43267 (local address 127.17.96.252:42201)
0619 02:29:57.713735 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:57.713741 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:57.714675 (+ 934us) server_negotiation.cc:407] Connection header received
0619 02:29:57.715495 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:57.715499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:57.715559 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:57.715624 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:57.717397 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.718367 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.720816 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.721312 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.721434 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.726594 (+ 5160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:57.726600 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:57.726603 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:57.726648 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:57.728852 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.730525 (+ 1673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.730533 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.730535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.730659 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.731173 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.731177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.731179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.731500 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:57.731607 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:57.732492 (+ 885us) server_negotiation.cc:299] Negotiation successful
0619 02:29:57.732654 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 02:29:57.732867 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.714966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:54431 (local address 127.17.96.253:32859)
0619 02:29:57.715131 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:57.715138 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:57.715156 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:29:57.715495 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:57.715499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:57.715559 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:57.715624 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:57.717384 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.718201 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.720842 (+ 2641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.721266 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.721369 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.727700 (+ 6331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:57.727708 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:57.727715 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:57.727757 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:57.730091 (+ 2334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.731168 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.731173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.731175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.731279 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.731663 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.731667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.731669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.731943 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:57.732040 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:57.732492 (+ 452us) server_negotiation.cc:299] Negotiation successful
0619 02:29:57.732654 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 02:29:57.733094 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.712744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:54223 (local address 127.17.96.254:33963)
0619 02:29:57.712939 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:57.712946 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:57.714233 (+ 1287us) server_negotiation.cc:407] Connection header received
0619 02:29:57.715487 (+ 1254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:57.715491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:57.715562 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:57.715684 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:57.717352 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.718379 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.720828 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.721266 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.721370 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.727662 (+ 6292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:57.727670 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:57.727677 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:57.727719 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:57.730277 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.731357 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.731364 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.731382 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.731485 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.731852 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.731859 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.731864 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.732166 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:57.732265 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:57.732708 (+ 443us) server_negotiation.cc:299] Negotiation successful
0619 02:29:57.732869 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 02:29:57.735041 25020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.714637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:32859 (local address 127.17.96.194:54431)
0619 02:29:57.714974 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:57.714988 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:57.715260 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:57.715894 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:57.715902 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:57.716469 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:57.717189 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.717201 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.718399 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.718402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.720665 (+ 2263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.720673 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.721670 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.721675 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.722264 (+ 589us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.723340 (+ 1076us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:57.723372 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:57.727454 (+ 4082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:57.730285 (+ 2831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.730289 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.730291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.731011 (+ 720us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.731426 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.731429 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.731432 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.731532 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.732130 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:57.732134 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:57.732224 (+ 90us) client_negotiation.cc:769] Sending connection context
0619 02:29:57.733852 (+ 1628us) client_negotiation.cc:240] Negotiation successful
0619 02:29:57.734027 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":1038,"spinlock_wait_cycles":50432,"thread_start_us":101,"threads_started":1}
I20260619 02:29:57.735067 25019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.713220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:42201 (local address 127.17.96.194:43267)
0619 02:29:57.714543 (+ 1323us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:57.714559 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:57.715259 (+ 700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:57.715894 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:57.715902 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:57.716466 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:57.717190 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.717201 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.718507 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.718511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.720619 (+ 2108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.720639 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.721671 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.721678 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.722246 (+ 568us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.723310 (+ 1064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:57.723341 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:57.726318 (+ 2977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:57.729039 (+ 2721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.729047 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.729107 (+ 60us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.730364 (+ 1257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.730806 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.730810 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.730812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.731042 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.731688 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:57.731696 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:57.732124 (+ 428us) client_negotiation.cc:769] Sending connection context
0619 02:29:57.733079 (+ 955us) client_negotiation.cc:240] Negotiation successful
0619 02:29:57.733649 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1258,"mutex_wait_us":646,"spinlock_wait_cycles":30080,"thread_start_us":209,"threads_started":1}
I20260619 02:29:57.735042 25018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.712871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.194:54223)
0619 02:29:57.713981 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:57.714046 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:57.715236 (+ 1190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:57.715891 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:57.715900 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:57.716463 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:57.717171 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.717187 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.718800 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.718803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.720634 (+ 1831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.720650 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.721670 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.721675 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.722251 (+ 576us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.723506 (+ 1255us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:57.723525 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:57.727440 (+ 3915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:57.730746 (+ 3306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.730752 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.730757 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.731216 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.731616 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.731622 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.731627 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.731728 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.732359 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:57.732362 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:57.732490 (+ 128us) client_negotiation.cc:769] Sending connection context
0619 02:29:57.733355 (+ 865us) client_negotiation.cc:240] Negotiation successful
0619 02:29:57.733802 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"mutex_wait_us":687,"spinlock_wait_cycles":11264,"thread_start_us":731,"threads_started":1}
I20260619 02:29:57.737133 25014 heartbeater.cc:347] Connected to a master server at 127.17.96.252:42201
I20260619 02:29:57.737186 25016 heartbeater.cc:347] Connected to a master server at 127.17.96.253:32859
I20260619 02:29:57.737448 25014 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:57.737447 25016 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:57.738075 25015 heartbeater.cc:347] Connected to a master server at 127.17.96.254:33963
I20260619 02:29:57.738179 25015 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:57.738353 25015 heartbeater.cc:511] Master 127.17.96.254:33963 requested a full tablet report, sending...
I20260619 02:29:57.738305 25014 heartbeater.cc:511] Master 127.17.96.252:42201 requested a full tablet report, sending...
I20260619 02:29:57.738307 25016 heartbeater.cc:511] Master 127.17.96.253:32859 requested a full tablet report, sending...
I20260619 02:29:57.739768 23907 ts_manager.cc:194] Re-registered known tserver with Master: b84f0480fc2c41e691849d8a341d0ffe (127.17.96.194:33775)
I20260619 02:29:57.740003 24499 ts_manager.cc:194] Registered new tserver with Master: b84f0480fc2c41e691849d8a341d0ffe (127.17.96.194:33775)
I20260619 02:29:57.740643 23907 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:54223
I20260619 02:29:57.741669 24693 ts_manager.cc:194] Registered new tserver with Master: b84f0480fc2c41e691849d8a341d0ffe (127.17.96.194:33775)
W20260619 02:29:57.865518 25021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:29:57.865942 25021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:29:57.866000 25021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:29:57.870851 25021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:29:57.870949 25021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:29:57.871021 25021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:29:57.877290 25021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:42083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.195.keytab
--principal=oryx/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:36321
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=44183
--tserver_master_addrs=127.17.96.254:33963,127.17.96.253:32859,127.17.96.252:42201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:29:57.878638 25021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:29:57.879827 25021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:29:57.891240 25029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:29:57.891259 25027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:57.891315 25021 server_base.cc:1034] running on GCE node
W20260619 02:29:57.891188 25026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:29:57.892119 25021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:29:57.892941 25021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:29:57.894145 25021 hybrid_clock.cc:648] HybridClock initialized: now 1781836197894113 us; error 64 us; skew 500 ppm
Jun 19 02:29:57 dist-test-slave-4qw6 krb5kdc[23862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836197, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:29:57.898154 25021 init.cc:377] Logged in from keytab as oryx/127.17.96.195@KRBTEST.COM (short username oryx)
I20260619 02:29:57.899565 25021 webserver.cc:460] Webserver started at http://127.17.96.195:44183/ using document root <none> and password file <none>
I20260619 02:29:57.900573 25021 fs_manager.cc:359] Metadata directory not provided
I20260619 02:29:57.900785 25021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:29:57.906090 25021 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:29:57.908937 25036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:57.910863 25021 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260619 02:29:57.911078 25021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "2d786c7d81db48f28b4cf8241046399f"
format_stamp: "Formatted at 2026-06-19 02:29:56 on dist-test-slave-4qw6"
server_key: "c73509d74a3a0739525414a22ec8b79f"
server_key_iv: "150d4915e97067b6154cd578b567316f"
server_key_version: "encryptionkey@0"
I20260619 02:29:57.911765 25021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:29:57.948378 25021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:29:57.951901 25021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:29:57.952149 25021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:29:57.952813 25021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:29:57.953853 25021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:29:57.953924 25021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:57.954002 25021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:29:57.954053 25021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:29:57.964632 25021 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:36321
I20260619 02:29:57.964668 25149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:36321 every 8 connection(s)
I20260619 02:29:57.965950 25021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:29:57.971036 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 25021
Jun 19 02:29:57 dist-test-slave-4qw6 krb5kdc[23862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836197, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.253@KRBTEST.COM
Jun 19 02:29:57 dist-test-slave-4qw6 krb5kdc[23862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836197, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM
Jun 19 02:29:57 dist-test-slave-4qw6 krb5kdc[23862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836197, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.252@KRBTEST.COM
I20260619 02:29:57.985971 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.969766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:57095 (local address 127.17.96.253:32859)
0619 02:29:57.969896 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:57.969899 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:57.970008 (+ 109us) server_negotiation.cc:407] Connection header received
0619 02:29:57.970505 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:57.970509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:57.970566 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:57.970624 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:57.972604 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.973451 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.975924 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.976280 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.976381 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.980865 (+ 4484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:57.980869 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:57.980872 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:57.980913 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:57.983238 (+ 2325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.983982 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.983987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.983990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.984086 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.984519 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.984522 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.984524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.984760 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:57.984843 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:57.985582 (+ 739us) server_negotiation.cc:299] Negotiation successful
0619 02:29:57.985778 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52,"spinlock_wait_cycles":2304}
I20260619 02:29:57.988086 25156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.969573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:32859 (local address 127.17.96.195:57095)
0619 02:29:57.969906 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:57.969921 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:57.970259 (+ 338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:57.970865 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:57.970873 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:57.971545 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:57.972405 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.972429 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.973642 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.973645 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.975706 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.975740 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.976672 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.976677 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.977008 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.977989 (+ 981us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:57.978019 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:57.980619 (+ 2600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:57.983392 (+ 2773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.983400 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.983426 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.983857 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.984211 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.984214 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.984216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.984411 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.984911 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:57.984918 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:57.985289 (+ 371us) client_negotiation.cc:769] Sending connection context
0619 02:29:57.986217 (+ 928us) client_negotiation.cc:240] Negotiation successful
0619 02:29:57.986761 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":174,"spinlock_wait_cycles":61696,"thread_start_us":119,"threads_started":1}
I20260619 02:29:57.988544 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.967938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:48229 (local address 127.17.96.252:42201)
0619 02:29:57.968080 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:57.968085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:57.969147 (+ 1062us) server_negotiation.cc:407] Connection header received
0619 02:29:57.970497 (+ 1350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:57.970501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:57.970565 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:57.970676 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:57.972604 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.973422 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.975914 (+ 2492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.976279 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.976399 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.983604 (+ 7205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:57.983611 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:57.983614 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:57.983648 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:57.985746 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.986692 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.986699 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.986703 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.986817 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.987163 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.987166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.987168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.987415 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:57.987523 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:57.988221 (+ 698us) server_negotiation.cc:299] Negotiation successful
0619 02:29:57.988366 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":58}
I20260619 02:29:57.989140 25155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.968418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:42201 (local address 127.17.96.195:48229)
0619 02:29:57.968961 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:57.969021 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:57.970269 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:57.970863 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:57.970871 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:57.971545 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:57.972423 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.972429 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.973612 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.973616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.975688 (+ 2072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.975708 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.978852 (+ 3144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.978859 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.979002 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.980257 (+ 1255us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:57.980279 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:57.982146 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:57.985948 (+ 3802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.985955 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.985960 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.986553 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.986941 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.986945 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.986947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.987045 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.987575 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:57.987578 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:57.987692 (+ 114us) client_negotiation.cc:769] Sending connection context
0619 02:29:57.987818 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 02:29:57.987986 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":406,"spinlock_wait_cycles":92032,"thread_start_us":200,"threads_started":1}
I20260619 02:29:57.990608 25152 heartbeater.cc:347] Connected to a master server at 127.17.96.253:32859
I20260619 02:29:57.990839 25150 heartbeater.cc:347] Connected to a master server at 127.17.96.252:42201
I20260619 02:29:57.990959 25152 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:57.990959 25150 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:57.991796 25150 heartbeater.cc:511] Master 127.17.96.252:42201 requested a full tablet report, sending...
I20260619 02:29:57.991796 25152 heartbeater.cc:511] Master 127.17.96.253:32859 requested a full tablet report, sending...
I20260619 02:29:57.993052 25161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.973314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41898 (local address 127.17.96.252:42201)
0619 02:29:57.974106 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:57.974112 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:57.974925 (+ 813us) server_negotiation.cc:407] Connection header received
0619 02:29:57.974975 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:57.974978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:57.975033 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:57.975124 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:57.976394 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.977355 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.982400 (+ 5045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.982856 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.983004 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.985716 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:57.985720 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:57.985723 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:57.985748 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:57.990108 (+ 4360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.990746 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.990753 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.990758 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.990925 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.991294 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.991299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.991301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.991577 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:57.991735 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:57.992624 (+ 889us) server_negotiation.cc:299] Negotiation successful
0619 02:29:57.992781 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":691,"thread_start_us":117,"threads_started":1}
I20260619 02:29:57.993227 24693 ts_manager.cc:194] Registered new tserver with Master: 2d786c7d81db48f28b4cf8241046399f (127.17.96.195:36321)
I20260619 02:29:57.993228 24499 ts_manager.cc:194] Registered new tserver with Master: 2d786c7d81db48f28b4cf8241046399f (127.17.96.195:36321)
I20260619 02:29:57.997422 25154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.968418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.195:51585)
0619 02:29:57.968962 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:57.969020 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:57.970259 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:57.971084 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:57.971087 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:57.971545 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:29:57.972407 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.972429 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.973615 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.973619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.977285 (+ 3666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:57.977303 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.978315 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.978320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:57.978449 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.979359 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:29:57.979381 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:29:57.981686 (+ 2305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:29:57.994937 (+ 13251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.994942 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.994945 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.995341 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.996008 (+ 667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:29:57.996012 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:29:57.996015 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:29:57.996144 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:29:57.996755 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:29:57.996758 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:29:57.996866 (+ 108us) client_negotiation.cc:769] Sending connection context
0619 02:29:57.996954 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 02:29:57.997173 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":2358,"spinlock_wait_cycles":25984,"thread_start_us":143,"threads_started":1}
I20260619 02:29:57.998500 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.967942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:51585 (local address 127.17.96.254:33963)
0619 02:29:57.968161 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:57.968166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:57.969149 (+ 983us) server_negotiation.cc:407] Connection header received
0619 02:29:57.970505 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:57.970509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:57.970579 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:57.970634 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:57.972603 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.973369 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.977510 (+ 4141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.978003 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.978135 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.982387 (+ 4252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:57.982392 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:57.982396 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:57.982436 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:57.994746 (+ 12310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.995545 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.995550 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.995552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.995688 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.996318 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.996324 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.996326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.996634 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:57.997856 (+ 1222us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:57.997959 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 02:29:57.998334 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260619 02:29:57.999295 25151 heartbeater.cc:347] Connected to a master server at 127.17.96.254:33963
I20260619 02:29:57.999511 25151 heartbeater.cc:464] Registering TS with master...
I20260619 02:29:57.999914 25151 heartbeater.cc:511] Master 127.17.96.254:33963 requested a full tablet report, sending...
I20260619 02:29:58.000207 25165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.975527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35650 (local address 127.17.96.253:32859)
0619 02:29:57.976270 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:57.976275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:57.976949 (+ 674us) server_negotiation.cc:407] Connection header received
0619 02:29:57.977103 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:57.977106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:57.977162 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:57.977260 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:57.978416 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.979178 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.980390 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.980733 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.980819 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:57.982898 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:57.982903 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:57.982906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:57.982937 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:57.994907 (+ 11970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.996554 (+ 1647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.996559 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.996561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.996950 (+ 389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:57.997393 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:57.997398 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:57.997402 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:57.997776 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:57.997957 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:57.999623 (+ 1666us) server_negotiation.cc:299] Negotiation successful
0619 02:29:57.999939 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":92,"threads_started":1}
I20260619 02:29:58.001070 23907 ts_manager.cc:194] Re-registered known tserver with Master: 2d786c7d81db48f28b4cf8241046399f (127.17.96.195:36321)
I20260619 02:29:58.001793 23907 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:51585
I20260619 02:29:58.006276 25163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:57.974260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51450 (local address 127.17.96.254:33963)
0619 02:29:57.983901 (+ 9641us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:57.983909 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:57.983932 (+ 23us) server_negotiation.cc:407] Connection header received
0619 02:29:57.983989 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:57.983995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:57.984059 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:57.984186 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:57.995045 (+ 10859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.995838 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.997428 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:57.998062 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:57.998184 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:58.001939 (+ 3755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:58.001946 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:58.001949 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:58.001985 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:58.004286 (+ 2301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:58.004853 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:58.004856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:58.004858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:58.004961 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:58.005201 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:58.005204 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:58.005205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:58.005423 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:58.005543 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:58.005846 (+ 303us) server_negotiation.cc:299] Negotiation successful
0619 02:29:58.005984 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9509,"thread_start_us":91,"threads_started":1}
I20260619 02:29:58.009373 23907 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51450:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:29:58.012849 23907 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:29:58.016574 24662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9556c261ac4a46dc97c064bef6d74c73" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: NON_VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: true } } }
I20260619 02:29:58.016799 24662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:58.019017 24662 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73: attempting to promote NON_VOTER 34f02c2e89484735991408738f401ee8 to VOTER
I20260619 02:29:58.019461 24662 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: false } }
I20260619 02:29:58.020874 24707 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 1 LEARNER]: Refusing update from remote peer 9556c261ac4a46dc97c064bef6d74c73: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260619 02:29:58.020884 24518 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 1 FOLLOWER]: Refusing update from remote peer 9556c261ac4a46dc97c064bef6d74c73: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260619 02:29:58.021271 25168 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260619 02:29:58.021482 25169 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260619 02:29:58.023605 24662 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 34f02c2e89484735991408738f401ee8 (127.17.96.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: false } } }
I20260619 02:29:58.023980 24707 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 34f02c2e89484735991408738f401ee8 (127.17.96.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: false } } }
I20260619 02:29:58.024034 24518 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 34f02c2e89484735991408738f401ee8 (127.17.96.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: false } } }
I20260619 02:29:58.025494 24662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9556c261ac4a46dc97c064bef6d74c73" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: false } } }
I20260619 02:29:58.025658 24662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:58.026427 25177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9556c261ac4a46dc97c064bef6d74c73" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: false } } }
I20260619 02:29:58.026605 25177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:29:58.027787 24740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9556c261ac4a46dc97c064bef6d74c73" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: false } } }
I20260619 02:29:58.027932 24740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:29:58.028714 24662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9556c261ac4a46dc97c064bef6d74c73" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: false } } }
I20260619 02:29:58.028846 24662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:58.029577 25169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9556c261ac4a46dc97c064bef6d74c73" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: false } } }
I20260619 02:29:58.029722 25169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73 [sys.catalog]: This master's current role is: LEADER
I20260619 02:29:58.039419 24464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:58.033140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:37611 (local address 127.0.0.1:43670)
0619 02:29:58.033300 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:58.033321 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:58.033426 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:58.034471 (+ 1045us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:58.034475 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:58.034498 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:58.034646 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:58.034655 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.036823 (+ 2168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.036829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:58.037708 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:58.037717 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.038888 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.038893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:58.039002 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:58.039036 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 02:29:58.039133 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 02:29:58.039217 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260619 02:29:58.039472 25179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:58.033354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43670 (local address 127.17.96.193:37611)
0619 02:29:58.033708 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:58.033715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:58.033737 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:29:58.033900 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:58.033906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:58.034250 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:58.034848 (+ 598us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:58.035020 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.036693 (+ 1673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.037899 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.038736 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.038858 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:58.039062 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:58.039165 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 02:29:58.039308 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":114,"threads_started":1}
I20260619 02:29:58.041478 25180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:58.033997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:36321 (local address 127.0.0.1:41320)
0619 02:29:58.034947 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:58.034965 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:58.035673 (+ 708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:58.036477 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:58.036480 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:58.036500 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:58.036623 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:58.036631 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.038338 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.038342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:58.039521 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:58.039536 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.041004 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.041007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:58.041084 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:58.041103 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 02:29:58.041215 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 02:29:58.041304 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"thread_start_us":116,"threads_started":1}
I20260619 02:29:58.041673 25181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:58.034233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41320 (local address 127.17.96.195:36321)
0619 02:29:58.034764 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:58.034781 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:58.035583 (+ 802us) server_negotiation.cc:407] Connection header received
0619 02:29:58.035873 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:58.035880 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:58.036121 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:58.036277 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:58.036776 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.038149 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.039698 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.040612 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.040720 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:58.040902 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:58.041254 (+ 352us) server_negotiation.cc:299] Negotiation successful
0619 02:29:58.041376 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":128,"threads_started":1}
I20260619 02:29:58.042377 25182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:58.034381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:33775 (local address 127.0.0.1:44342)
0619 02:29:58.034988 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:58.035010 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:58.035655 (+ 645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:58.036648 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:58.036650 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:58.036666 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:58.036749 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:58.036754 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.038323 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.038327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:58.040334 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:58.040345 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.041877 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.041881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:58.041978 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:58.042015 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 02:29:58.042103 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 02:29:58.042236 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"mutex_wait_us":958,"thread_start_us":111,"threads_started":1}
I20260619 02:29:58.042477 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:58.034579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44342 (local address 127.17.96.194:33775)
0619 02:29:58.035939 (+ 1360us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:58.035954 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:58.035976 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:29:58.036078 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:58.036083 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:58.036307 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:58.036470 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:58.036902 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.038184 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.040491 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.041294 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.041411 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:58.041554 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:58.042170 (+ 616us) server_negotiation.cc:299] Negotiation successful
0619 02:29:58.042293 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1243,"thread_start_us":96,"threads_started":1}
I20260619 02:29:58.043504 24811 tablet_service.cc:1467] Processing CreateTablet for tablet 191bbb75c35d46caa3d8a9b916092022 (DEFAULT_TABLE table=test-table [id=fde1e34051504d9bb84c51af18c98772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:58.044723 24811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 191bbb75c35d46caa3d8a9b916092022. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:58.044746 25084 tablet_service.cc:1467] Processing CreateTablet for tablet 191bbb75c35d46caa3d8a9b916092022 (DEFAULT_TABLE table=test-table [id=fde1e34051504d9bb84c51af18c98772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:58.045879 25084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 191bbb75c35d46caa3d8a9b916092022. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:58.049163 24947 tablet_service.cc:1467] Processing CreateTablet for tablet 191bbb75c35d46caa3d8a9b916092022 (DEFAULT_TABLE table=test-table [id=fde1e34051504d9bb84c51af18c98772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:29:58.050590 24947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 191bbb75c35d46caa3d8a9b916092022. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:29:58.053341 25184 tablet_bootstrap.cc:492] T 191bbb75c35d46caa3d8a9b916092022 P 153c108b8f4f4fce97375af0dcae499b: Bootstrap starting.
I20260619 02:29:58.054826 25185 tablet_bootstrap.cc:492] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f: Bootstrap starting.
I20260619 02:29:58.056205 25184 tablet_bootstrap.cc:654] T 191bbb75c35d46caa3d8a9b916092022 P 153c108b8f4f4fce97375af0dcae499b: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:58.057183 25185 tablet_bootstrap.cc:654] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:58.057355 25184 log.cc:824] T 191bbb75c35d46caa3d8a9b916092022 P 153c108b8f4f4fce97375af0dcae499b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:58.058331 25185 log.cc:824] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:58.060035 25184 tablet_bootstrap.cc:492] T 191bbb75c35d46caa3d8a9b916092022 P 153c108b8f4f4fce97375af0dcae499b: No bootstrap required, opened a new log
I20260619 02:29:58.060273 25184 ts_tablet_manager.cc:1397] T 191bbb75c35d46caa3d8a9b916092022 P 153c108b8f4f4fce97375af0dcae499b: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260619 02:29:58.060464 25188 tablet_bootstrap.cc:492] T 191bbb75c35d46caa3d8a9b916092022 P b84f0480fc2c41e691849d8a341d0ffe: Bootstrap starting.
I20260619 02:29:58.060647 25185 tablet_bootstrap.cc:492] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f: No bootstrap required, opened a new log
I20260619 02:29:58.060848 25185 ts_tablet_manager.cc:1397] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260619 02:29:58.063373 25188 tablet_bootstrap.cc:654] T 191bbb75c35d46caa3d8a9b916092022 P b84f0480fc2c41e691849d8a341d0ffe: Neither blocks nor log segments found. Creating new log.
I20260619 02:29:58.064355 25188 log.cc:824] T 191bbb75c35d46caa3d8a9b916092022 P b84f0480fc2c41e691849d8a341d0ffe: Log is configured to *not* fsync() on all Append() calls
I20260619 02:29:58.065006 25184 raft_consensus.cc:348] T 191bbb75c35d46caa3d8a9b916092022 P 153c108b8f4f4fce97375af0dcae499b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153c108b8f4f4fce97375af0dcae499b" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37611 } } peers { permanent_uuid: "2d786c7d81db48f28b4cf8241046399f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36321 } } peers { permanent_uuid: "b84f0480fc2c41e691849d8a341d0ffe" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33775 } }
I20260619 02:29:58.065358 25184 raft_consensus.cc:374] T 191bbb75c35d46caa3d8a9b916092022 P 153c108b8f4f4fce97375af0dcae499b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:58.065434 25184 raft_consensus.cc:729] T 191bbb75c35d46caa3d8a9b916092022 P 153c108b8f4f4fce97375af0dcae499b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 153c108b8f4f4fce97375af0dcae499b, State: Initialized, Role: FOLLOWER
I20260619 02:29:58.065735 25185 raft_consensus.cc:348] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153c108b8f4f4fce97375af0dcae499b" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37611 } } peers { permanent_uuid: "2d786c7d81db48f28b4cf8241046399f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36321 } } peers { permanent_uuid: "b84f0480fc2c41e691849d8a341d0ffe" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33775 } }
I20260619 02:29:58.066042 25185 raft_consensus.cc:374] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:58.066103 25185 raft_consensus.cc:729] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d786c7d81db48f28b4cf8241046399f, State: Initialized, Role: FOLLOWER
I20260619 02:29:58.066058 25184 consensus_queue.cc:260] T 191bbb75c35d46caa3d8a9b916092022 P 153c108b8f4f4fce97375af0dcae499b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153c108b8f4f4fce97375af0dcae499b" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37611 } } peers { permanent_uuid: "2d786c7d81db48f28b4cf8241046399f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36321 } } peers { permanent_uuid: "b84f0480fc2c41e691849d8a341d0ffe" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33775 } }
I20260619 02:29:58.066543 25185 consensus_queue.cc:260] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153c108b8f4f4fce97375af0dcae499b" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37611 } } peers { permanent_uuid: "2d786c7d81db48f28b4cf8241046399f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36321 } } peers { permanent_uuid: "b84f0480fc2c41e691849d8a341d0ffe" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33775 } }
I20260619 02:29:58.067044 24878 heartbeater.cc:503] Master 127.17.96.254:33963 was elected leader, sending a full tablet report...
I20260619 02:29:58.067309 25188 tablet_bootstrap.cc:492] T 191bbb75c35d46caa3d8a9b916092022 P b84f0480fc2c41e691849d8a341d0ffe: No bootstrap required, opened a new log
I20260619 02:29:58.067538 25188 ts_tablet_manager.cc:1397] T 191bbb75c35d46caa3d8a9b916092022 P b84f0480fc2c41e691849d8a341d0ffe: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260619 02:29:58.067670 25185 ts_tablet_manager.cc:1428] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260619 02:29:58.068869 25151 heartbeater.cc:503] Master 127.17.96.254:33963 was elected leader, sending a full tablet report...
I20260619 02:29:58.070422 25184 ts_tablet_manager.cc:1428] T 191bbb75c35d46caa3d8a9b916092022 P 153c108b8f4f4fce97375af0dcae499b: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260619 02:29:58.078336 25188 raft_consensus.cc:348] T 191bbb75c35d46caa3d8a9b916092022 P b84f0480fc2c41e691849d8a341d0ffe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153c108b8f4f4fce97375af0dcae499b" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37611 } } peers { permanent_uuid: "2d786c7d81db48f28b4cf8241046399f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36321 } } peers { permanent_uuid: "b84f0480fc2c41e691849d8a341d0ffe" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33775 } }
I20260619 02:29:58.078657 25188 raft_consensus.cc:374] T 191bbb75c35d46caa3d8a9b916092022 P b84f0480fc2c41e691849d8a341d0ffe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:29:58.078706 25188 raft_consensus.cc:729] T 191bbb75c35d46caa3d8a9b916092022 P b84f0480fc2c41e691849d8a341d0ffe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b84f0480fc2c41e691849d8a341d0ffe, State: Initialized, Role: FOLLOWER
I20260619 02:29:58.079196 25188 consensus_queue.cc:260] T 191bbb75c35d46caa3d8a9b916092022 P b84f0480fc2c41e691849d8a341d0ffe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153c108b8f4f4fce97375af0dcae499b" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37611 } } peers { permanent_uuid: "2d786c7d81db48f28b4cf8241046399f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36321 } } peers { permanent_uuid: "b84f0480fc2c41e691849d8a341d0ffe" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33775 } }
I20260619 02:29:58.080399 25015 heartbeater.cc:503] Master 127.17.96.254:33963 was elected leader, sending a full tablet report...
I20260619 02:29:58.080495 25188 ts_tablet_manager.cc:1428] T 191bbb75c35d46caa3d8a9b916092022 P b84f0480fc2c41e691849d8a341d0ffe: Time spent starting tablet: real 0.013s user 0.007s sys 0.000s
W20260619 02:29:58.212191 25017 tablet.cc:2369] T 191bbb75c35d46caa3d8a9b916092022 P b84f0480fc2c41e691849d8a341d0ffe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:29:58.218181 25153 tablet.cc:2369] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:29:58.222734 24880 tablet.cc:2369] T 191bbb75c35d46caa3d8a9b916092022 P 153c108b8f4f4fce97375af0dcae499b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:29:58.253039 25191 raft_consensus.cc:482] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:29:58.253266 25191 raft_consensus.cc:504] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153c108b8f4f4fce97375af0dcae499b" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37611 } } peers { permanent_uuid: "2d786c7d81db48f28b4cf8241046399f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36321 } } peers { permanent_uuid: "b84f0480fc2c41e691849d8a341d0ffe" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33775 } }
I20260619 02:29:58.254691 25191 leader_election.cc:288] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 153c108b8f4f4fce97375af0dcae499b (127.17.96.193:37611), b84f0480fc2c41e691849d8a341d0ffe (127.17.96.194:33775)
I20260619 02:29:58.261369 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:58.255133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:51719 (local address 127.17.96.194:33775)
0619 02:29:58.255274 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:58.255279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:58.255325 (+ 46us) server_negotiation.cc:407] Connection header received
0619 02:29:58.255473 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:58.255477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:58.255535 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:58.255634 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:58.256149 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.257038 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.259676 (+ 2638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.260407 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.260586 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:58.260649 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:58.261097 (+ 448us) server_negotiation.cc:299] Negotiation successful
0619 02:29:58.261193 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 02:29:58.261369 25179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:58.255033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:51237 (local address 127.17.96.193:37611)
0619 02:29:58.255164 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:58.255170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:58.255277 (+ 107us) server_negotiation.cc:407] Connection header received
0619 02:29:58.255473 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:58.255477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:58.255535 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:58.255634 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:58.256149 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.257038 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.259676 (+ 2638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.260456 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.260586 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:58.260649 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:58.261097 (+ 448us) server_negotiation.cc:299] Negotiation successful
0619 02:29:58.261193 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:29:58.261435 25155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:58.255068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:33775 (local address 127.17.96.195:51719)
0619 02:29:58.255266 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:58.255279 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:58.255348 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:58.255667 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:58.255671 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:58.255705 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:58.256012 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:58.256021 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.257290 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.257296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:58.259431 (+ 2135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:58.259456 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.260621 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.260626 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:58.260852 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:58.260939 (+ 87us) client_negotiation.cc:769] Sending connection context
0619 02:29:58.261061 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 02:29:58.261199 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":505,"spinlock_wait_cycles":15616}
I20260619 02:29:58.261435 25154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:58.254946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:37611 (local address 127.17.96.195:51237)
0619 02:29:58.255162 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:58.255179 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:58.255310 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:58.255667 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:58.255671 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:58.255703 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:58.256012 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:58.256021 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.257290 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.257296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:58.259431 (+ 2135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:58.259456 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.260621 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.260626 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:58.260851 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:58.260940 (+ 89us) client_negotiation.cc:769] Sending connection context
0619 02:29:58.261056 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 02:29:58.261199 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":399,"spinlock_wait_cycles":6656}
I20260619 02:29:58.262559 24831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "191bbb75c35d46caa3d8a9b916092022" candidate_uuid: "2d786c7d81db48f28b4cf8241046399f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "153c108b8f4f4fce97375af0dcae499b" is_pre_election: true
I20260619 02:29:58.262614 24965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "191bbb75c35d46caa3d8a9b916092022" candidate_uuid: "2d786c7d81db48f28b4cf8241046399f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b84f0480fc2c41e691849d8a341d0ffe" is_pre_election: true
I20260619 02:29:58.262969 24831 raft_consensus.cc:2454] T 191bbb75c35d46caa3d8a9b916092022 P 153c108b8f4f4fce97375af0dcae499b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d786c7d81db48f28b4cf8241046399f in term 0.
I20260619 02:29:58.262977 24965 raft_consensus.cc:2454] T 191bbb75c35d46caa3d8a9b916092022 P b84f0480fc2c41e691849d8a341d0ffe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d786c7d81db48f28b4cf8241046399f in term 0.
I20260619 02:29:58.263680 25038 leader_election.cc:302] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 153c108b8f4f4fce97375af0dcae499b, 2d786c7d81db48f28b4cf8241046399f, b84f0480fc2c41e691849d8a341d0ffe; no voters:
I20260619 02:29:58.264003 25191 raft_consensus.cc:2781] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:29:58.264117 25191 raft_consensus.cc:482] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:29:58.264185 25191 raft_consensus.cc:3037] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:58.265483 25191 raft_consensus.cc:504] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153c108b8f4f4fce97375af0dcae499b" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37611 } } peers { permanent_uuid: "2d786c7d81db48f28b4cf8241046399f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36321 } } peers { permanent_uuid: "b84f0480fc2c41e691849d8a341d0ffe" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33775 } }
I20260619 02:29:58.266063 25191 leader_election.cc:288] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f [CANDIDATE]: Term 1 election: Requested vote from peers 153c108b8f4f4fce97375af0dcae499b (127.17.96.193:37611), b84f0480fc2c41e691849d8a341d0ffe (127.17.96.194:33775)
I20260619 02:29:58.266556 24831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "191bbb75c35d46caa3d8a9b916092022" candidate_uuid: "2d786c7d81db48f28b4cf8241046399f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "153c108b8f4f4fce97375af0dcae499b"
I20260619 02:29:58.266691 24965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "191bbb75c35d46caa3d8a9b916092022" candidate_uuid: "2d786c7d81db48f28b4cf8241046399f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b84f0480fc2c41e691849d8a341d0ffe"
I20260619 02:29:58.266767 24831 raft_consensus.cc:3037] T 191bbb75c35d46caa3d8a9b916092022 P 153c108b8f4f4fce97375af0dcae499b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:58.266834 24965 raft_consensus.cc:3037] T 191bbb75c35d46caa3d8a9b916092022 P b84f0480fc2c41e691849d8a341d0ffe [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:29:58.267917 24831 raft_consensus.cc:2454] T 191bbb75c35d46caa3d8a9b916092022 P 153c108b8f4f4fce97375af0dcae499b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d786c7d81db48f28b4cf8241046399f in term 1.
I20260619 02:29:58.267937 24965 raft_consensus.cc:2454] T 191bbb75c35d46caa3d8a9b916092022 P b84f0480fc2c41e691849d8a341d0ffe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d786c7d81db48f28b4cf8241046399f in term 1.
I20260619 02:29:58.268296 25039 leader_election.cc:302] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 153c108b8f4f4fce97375af0dcae499b, 2d786c7d81db48f28b4cf8241046399f; no voters:
I20260619 02:29:58.268519 25191 raft_consensus.cc:2781] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:29:58.268774 25191 raft_consensus.cc:686] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f [term 1 LEADER]: Becoming Leader. State: Replica: 2d786c7d81db48f28b4cf8241046399f, State: Running, Role: LEADER
I20260619 02:29:58.269224 25191 consensus_queue.cc:237] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153c108b8f4f4fce97375af0dcae499b" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37611 } } peers { permanent_uuid: "2d786c7d81db48f28b4cf8241046399f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36321 } } peers { permanent_uuid: "b84f0480fc2c41e691849d8a341d0ffe" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33775 } }
I20260619 02:29:58.274560 23907 catalog_manager.cc:5495] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d786c7d81db48f28b4cf8241046399f (127.17.96.195). New cstate: current_term: 1 leader_uuid: "2d786c7d81db48f28b4cf8241046399f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153c108b8f4f4fce97375af0dcae499b" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d786c7d81db48f28b4cf8241046399f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b84f0480fc2c41e691849d8a341d0ffe" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33775 } health_report { overall_health: UNKNOWN } } }
I20260619 02:29:58.347213 25181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:58.343172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41332 (local address 127.17.96.195:36321)
0619 02:29:58.343358 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:58.343363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:58.343404 (+ 41us) server_negotiation.cc:407] Connection header received
0619 02:29:58.343520 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:58.343523 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:58.343634 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:58.343791 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:29:58.344114 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.344667 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.345655 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.345940 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.346034 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:58.346328 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:29:58.346816 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:29:58.346916 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:58.346996 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 02:29:58.347055 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 02:29:58.354076 24965 raft_consensus.cc:1261] T 191bbb75c35d46caa3d8a9b916092022 P b84f0480fc2c41e691849d8a341d0ffe [term 1 FOLLOWER]: Refusing update from remote peer 2d786c7d81db48f28b4cf8241046399f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:29:58.354076 24831 raft_consensus.cc:1261] T 191bbb75c35d46caa3d8a9b916092022 P 153c108b8f4f4fce97375af0dcae499b [term 1 FOLLOWER]: Refusing update from remote peer 2d786c7d81db48f28b4cf8241046399f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:29:58.354944 25191 consensus_queue.cc:1035] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b84f0480fc2c41e691849d8a341d0ffe" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:29:58.355263 25193 consensus_queue.cc:1035] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f [LEADER]: Connected to new peer: Peer: permanent_uuid: "153c108b8f4f4fce97375af0dcae499b" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:29:58.366752 25198 mvcc.cc:205] Tried to move back new op lower bound from 7298401068449869824 to 7298401068114382848. Current Snapshot: MvccSnapshot[applied={T|T < 7298401068449869824}]
I20260619 02:29:58.380544 25197 mvcc.cc:205] Tried to move back new op lower bound from 7298401068449869824 to 7298401068114382848. Current Snapshot: MvccSnapshot[applied={T|T < 7298401068449869824 or (T in {7298401068449869824})}]
I20260619 02:29:58.381969 23905 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:29:58.382194 23905 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:29:58.387344 23905 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9556c261ac4a46dc97c064bef6d74c73: Sending DeleteTablet for 3 replicas of tablet 191bbb75c35d46caa3d8a9b916092022
I20260619 02:29:58.388607 25084 tablet_service.cc:1514] Processing DeleteTablet for tablet 191bbb75c35d46caa3d8a9b916092022 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:58 UTC) from {username='oryx', principal='oryx/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:41320
I20260619 02:29:58.388608 24947 tablet_service.cc:1514] Processing DeleteTablet for tablet 191bbb75c35d46caa3d8a9b916092022 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:58 UTC) from {username='oryx', principal='oryx/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:44342
I20260619 02:29:58.389432 24811 tablet_service.cc:1514] Processing DeleteTablet for tablet 191bbb75c35d46caa3d8a9b916092022 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:29:58 UTC) from {username='oryx', principal='oryx/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:43670
I20260619 02:29:58.390224 25206 tablet_replica.cc:330] T 191bbb75c35d46caa3d8a9b916092022 P b84f0480fc2c41e691849d8a341d0ffe: stopping tablet replica
I20260619 02:29:58.390623 25206 raft_consensus.cc:2229] T 191bbb75c35d46caa3d8a9b916092022 P b84f0480fc2c41e691849d8a341d0ffe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:29:58.391019 25206 raft_consensus.cc:2258] T 191bbb75c35d46caa3d8a9b916092022 P b84f0480fc2c41e691849d8a341d0ffe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:58.391801 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 23875
I20260619 02:29:58.393708 25206 ts_tablet_manager.cc:1905] T 191bbb75c35d46caa3d8a9b916092022 P b84f0480fc2c41e691849d8a341d0ffe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:58.397924 25206 ts_tablet_manager.cc:1918] T 191bbb75c35d46caa3d8a9b916092022 P b84f0480fc2c41e691849d8a341d0ffe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:29:58.398068 25206 log.cc:1196] T 191bbb75c35d46caa3d8a9b916092022 P b84f0480fc2c41e691849d8a341d0ffe: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/191bbb75c35d46caa3d8a9b916092022
I20260619 02:29:58.398458 25206 ts_tablet_manager.cc:1939] T 191bbb75c35d46caa3d8a9b916092022 P b84f0480fc2c41e691849d8a341d0ffe: Deleting consensus metadata
I20260619 02:29:58.400401 25209 tablet_replica.cc:330] T 191bbb75c35d46caa3d8a9b916092022 P 153c108b8f4f4fce97375af0dcae499b: stopping tablet replica
W20260619 02:29:58.400496 24904 connection.cc:535] server connection from 127.0.0.1:44342 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 02:29:58.401480 25209 raft_consensus.cc:2229] T 191bbb75c35d46caa3d8a9b916092022 P 153c108b8f4f4fce97375af0dcae499b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:29:58.401950 25209 raft_consensus.cc:2258] T 191bbb75c35d46caa3d8a9b916092022 P 153c108b8f4f4fce97375af0dcae499b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:58.405094 25209 ts_tablet_manager.cc:1905] T 191bbb75c35d46caa3d8a9b916092022 P 153c108b8f4f4fce97375af0dcae499b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:29:58.408694 25207 tablet_replica.cc:330] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f: stopping tablet replica
I20260619 02:29:58.408876 25209 ts_tablet_manager.cc:1918] T 191bbb75c35d46caa3d8a9b916092022 P 153c108b8f4f4fce97375af0dcae499b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:29:58.409060 25209 log.cc:1196] T 191bbb75c35d46caa3d8a9b916092022 P 153c108b8f4f4fce97375af0dcae499b: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-0/wal/wals/191bbb75c35d46caa3d8a9b916092022
I20260619 02:29:58.409162 25207 raft_consensus.cc:2229] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:29:58.409454 25209 ts_tablet_manager.cc:1939] T 191bbb75c35d46caa3d8a9b916092022 P 153c108b8f4f4fce97375af0dcae499b: Deleting consensus metadata
I20260619 02:29:58.409607 25207 raft_consensus.cc:2258] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:29:58.412326 25207 ts_tablet_manager.cc:1905] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 02:29:58.412604 24765 connection.cc:441] server connection from 127.0.0.1:43670 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43670 (request call id 1) could send its response
I20260619 02:29:58.417922 25207 ts_tablet_manager.cc:1918] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:29:58.418103 25207 log.cc:1196] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/191bbb75c35d46caa3d8a9b916092022
I20260619 02:29:58.418548 25207 ts_tablet_manager.cc:1939] T 191bbb75c35d46caa3d8a9b916092022 P 2d786c7d81db48f28b4cf8241046399f: Deleting consensus metadata
W20260619 02:29:58.420110 25037 connection.cc:441] server connection from 127.0.0.1:41320 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41320 (request call id 1) could send its response
I20260619 02:29:58.422735 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:58.407768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35666 (local address 127.17.96.253:32859)
0619 02:29:58.407954 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:58.407959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:58.410581 (+ 2622us) server_negotiation.cc:407] Connection header received
0619 02:29:58.410791 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:58.410797 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:58.410880 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:58.410983 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:58.412598 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.413457 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.415070 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.415427 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.415516 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:58.417662 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:58.417667 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:58.417670 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:58.417705 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:58.420524 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:58.421081 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:58.421085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:58.421088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:58.421185 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:58.421531 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:58.421535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:58.421537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:58.421835 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:58.421964 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:58.422391 (+ 427us) server_negotiation.cc:299] Negotiation successful
0619 02:29:58.422542 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260619 02:29:58.423400 25165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:58.390162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35652 (local address 127.17.96.253:32859)
0619 02:29:58.390298 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:58.390303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:58.402571 (+ 12268us) server_negotiation.cc:407] Connection header received
0619 02:29:58.402808 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:58.402811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:58.402870 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:58.402932 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:58.404462 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.405427 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.407090 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.407506 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.407593 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:58.417601 (+ 10008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:58.417605 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:58.417609 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:58.417640 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:58.420169 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:58.420883 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:58.420888 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:58.420890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:58.420996 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:58.421377 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:58.421383 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:58.421386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:58.421682 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:58.422042 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:58.422540 (+ 498us) server_negotiation.cc:299] Negotiation successful
0619 02:29:58.422700 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 02:29:58.426553 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:58.406253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41912 (local address 127.17.96.252:42201)
0619 02:29:58.406398 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:58.406403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:58.410097 (+ 3694us) server_negotiation.cc:407] Connection header received
0619 02:29:58.410251 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:58.410256 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:58.410321 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:58.410428 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:58.412840 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.413615 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.415903 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.416290 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.416420 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:58.422101 (+ 5681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:58.422108 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:58.422115 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:58.422159 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:58.424426 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:58.424986 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:58.424990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:58.424992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:58.425092 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:58.425435 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:58.425439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:58.425441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:58.425655 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:58.425816 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:58.426204 (+ 388us) server_negotiation.cc:299] Negotiation successful
0619 02:29:58.426354 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 02:29:58.433038 25161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:58.390203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41900 (local address 127.17.96.252:42201)
0619 02:29:58.390342 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:58.390347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:58.423433 (+ 33086us) server_negotiation.cc:407] Connection header received
0619 02:29:58.423666 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:58.423670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:58.423739 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:58.423808 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:29:58.424765 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.425681 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.427187 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:58.427573 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:58.427731 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:58.429367 (+ 1636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:29:58.429371 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:29:58.429374 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:29:58.429410 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:29:58.431111 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:58.431601 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:58.431605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:58.431607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:58.431707 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:29:58.432022 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:29:58.432028 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:29:58.432030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:29:58.432245 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:29:58.432418 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:58.432684 (+ 266us) server_negotiation.cc:299] Negotiation successful
0619 02:29:58.432828 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 02:29:59.370895 25217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:59.370342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.193:58785)
0619 02:29:59.370678 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:59.370771 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":83,"threads_started":1}
W20260619 02:29:59.371285 24878 heartbeater.cc:644] Failed to heartbeat to 127.17.96.254:33963 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
I20260619 02:29:59.375612 25219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:59.375066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.194:45541)
0619 02:29:59.375372 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:59.375488 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":84,"threads_started":1}
W20260619 02:29:59.375994 25015 heartbeater.cc:644] Failed to heartbeat to 127.17.96.254:33963 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
I20260619 02:29:59.378314 25218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:59.374778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.195:53357)
0619 02:29:59.378025 (+ 3247us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:59.378168 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3124,"thread_start_us":103,"threads_started":1}
W20260619 02:29:59.378733 25151 heartbeater.cc:644] Failed to heartbeat to 127.17.96.254:33963 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
I20260619 02:29:59.888159 25220 raft_consensus.cc:482] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9556c261ac4a46dc97c064bef6d74c73)
I20260619 02:29:59.888463 25220 raft_consensus.cc:504] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: false } }
I20260619 02:29:59.888739 25221 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9556c261ac4a46dc97c064bef6d74c73)
I20260619 02:29:59.888991 25221 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: false } }
I20260619 02:29:59.889818 25220 leader_election.cc:288] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9556c261ac4a46dc97c064bef6d74c73 (127.17.96.254:33963), 34f02c2e89484735991408738f401ee8 (127.17.96.252:42201)
I20260619 02:29:59.889997 25221 leader_election.cc:288] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9556c261ac4a46dc97c064bef6d74c73 (127.17.96.254:33963), 549beb78d8c6431184a1e0282c52e6f0 (127.17.96.253:32859)
I20260619 02:29:59.890695 25222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:59.890038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.0.0.1:51964)
0619 02:29:59.890441 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:59.890545 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":92,"threads_started":1}
I20260619 02:29:59.890895 25225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:59.890455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.0.0.1:51976)
0619 02:29:59.890685 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:59.890773 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":60,"threads_started":1}
W20260619 02:29:59.890903 24495 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
W20260619 02:29:59.891340 24685 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
I20260619 02:29:59.892333 25222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:59.891986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.0.0.1:51990)
0619 02:29:59.892169 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:59.892226 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260619 02:29:59.892467 24495 leader_election.cc:334] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9556c261ac4a46dc97c064bef6d74c73 (127.17.96.254:33963): Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
I20260619 02:29:59.893417 25229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:59.892118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.0.0.1:52006)
0619 02:29:59.893188 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:59.893255 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":992,"thread_start_us":94,"threads_started":1}
W20260619 02:29:59.893540 24685 leader_election.cc:334] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9556c261ac4a46dc97c064bef6d74c73 (127.17.96.254:33963): Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
I20260619 02:29:59.896024 25224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:59.890285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41914 (local address 127.17.96.252:42201)
0619 02:29:59.890521 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:59.890528 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:59.890552 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:29:59.890782 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:59.890786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:59.890845 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:59.890917 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:59.891490 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:59.892376 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:59.894224 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:59.895052 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:59.895220 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:59.895322 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:59.895676 (+ 354us) server_negotiation.cc:299] Negotiation successful
0619 02:29:59.895794 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":161,"thread_start_us":69,"threads_started":1}
I20260619 02:29:59.896044 25223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:59.890040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:42201 (local address 127.0.0.1:41914)
0619 02:29:59.890471 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:59.890486 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:59.890615 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:59.891100 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:59.891104 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:59.891129 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:59.891284 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:59.891295 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:59.892544 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:59.892551 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:59.893923 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:59.893936 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:59.895253 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:59.895256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:59.895476 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:59.895526 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 02:29:59.895627 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 02:29:59.895844 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":141,"thread_start_us":59,"threads_started":1}
I20260619 02:29:59.896315 25225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:59.891159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:32859 (local address 127.0.0.1:35672)
0619 02:29:59.891243 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:59.891257 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:29:59.891356 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:29:59.891667 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:29:59.891673 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:29:59.891695 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:29:59.891971 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:59.891982 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:59.893064 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:59.893069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:59.894516 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:29:59.894527 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:29:59.895725 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:29:59.895730 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:29:59.895850 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:59.895884 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 02:29:59.895998 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 02:29:59.896109 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":151}
I20260619 02:29:59.896665 24707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "549beb78d8c6431184a1e0282c52e6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "34f02c2e89484735991408738f401ee8" is_pre_election: true
I20260619 02:29:59.896752 25226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:59.890811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35672 (local address 127.17.96.253:32859)
0619 02:29:59.891127 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0619 02:29:59.891132 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:29:59.891449 (+ 317us) server_negotiation.cc:407] Connection header received
0619 02:29:59.891499 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:29:59.891502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:29:59.891547 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:29:59.891808 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:29:59.892129 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:59.892924 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:59.894692 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:29:59.895555 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:29:59.895702 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:29:59.895772 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:29:59.896034 (+ 262us) server_negotiation.cc:299] Negotiation successful
0619 02:29:59.896588 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":250,"thread_start_us":72,"threads_started":1}
I20260619 02:29:59.896950 24707 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 549beb78d8c6431184a1e0282c52e6f0 in term 1.
I20260619 02:29:59.897300 24518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34f02c2e89484735991408738f401ee8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "549beb78d8c6431184a1e0282c52e6f0" is_pre_election: true
I20260619 02:29:59.897477 24492 leader_election.cc:302] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f02c2e89484735991408738f401ee8, 549beb78d8c6431184a1e0282c52e6f0; no voters: 9556c261ac4a46dc97c064bef6d74c73
I20260619 02:29:59.897603 24518 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f02c2e89484735991408738f401ee8 in term 1.
I20260619 02:29:59.897883 25220 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 02:29:59.897980 25220 raft_consensus.cc:482] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9556c261ac4a46dc97c064bef6d74c73)
I20260619 02:29:59.898036 25220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:29:59.898239 24685 leader_election.cc:302] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f02c2e89484735991408738f401ee8, 549beb78d8c6431184a1e0282c52e6f0; no voters: 9556c261ac4a46dc97c064bef6d74c73
I20260619 02:29:59.898666 25221 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 02:29:59.898736 25221 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9556c261ac4a46dc97c064bef6d74c73)
I20260619 02:29:59.898772 25221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:29:59.899657 25220 raft_consensus.cc:504] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: false } }
I20260619 02:29:59.900180 25220 leader_election.cc:288] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [CANDIDATE]: Term 2 election: Requested vote from peers 9556c261ac4a46dc97c064bef6d74c73 (127.17.96.254:33963), 34f02c2e89484735991408738f401ee8 (127.17.96.252:42201)
I20260619 02:29:59.900072 25221 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: false } }
I20260619 02:29:59.900695 25221 leader_election.cc:288] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [CANDIDATE]: Term 2 election: Requested vote from peers 9556c261ac4a46dc97c064bef6d74c73 (127.17.96.254:33963), 549beb78d8c6431184a1e0282c52e6f0 (127.17.96.253:32859)
I20260619 02:29:59.900810 24707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "549beb78d8c6431184a1e0282c52e6f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "34f02c2e89484735991408738f401ee8"
I20260619 02:29:59.901052 24707 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 549beb78d8c6431184a1e0282c52e6f0 in current term 2: Already voted for candidate 34f02c2e89484735991408738f401ee8 in this term.
I20260619 02:29:59.901178 25223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:59.900405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.0.0.1:52022)
0619 02:29:59.900607 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:59.901088 (+ 481us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260619 02:29:59.901185 25225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:59.900823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.0.0.1:52026)
0619 02:29:59.900987 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:59.901072 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 02:29:59.901338 24518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34f02c2e89484735991408738f401ee8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "549beb78d8c6431184a1e0282c52e6f0"
I20260619 02:29:59.901595 24518 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 34f02c2e89484735991408738f401ee8 in current term 2: Already voted for candidate 549beb78d8c6431184a1e0282c52e6f0 in this term.
I20260619 02:29:59.902144 25225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:59.901864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.0.0.1:52030)
0619 02:29:59.901992 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:59.902054 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260619 02:29:59.902279 24685 leader_election.cc:334] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9556c261ac4a46dc97c064bef6d74c73 (127.17.96.254:33963): Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
I20260619 02:29:59.902344 24685 leader_election.cc:302] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34f02c2e89484735991408738f401ee8; no voters: 549beb78d8c6431184a1e0282c52e6f0, 9556c261ac4a46dc97c064bef6d74c73
I20260619 02:29:59.902539 25223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:29:59.902225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.0.0.1:52044)
0619 02:29:59.902357 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 02:29:59.902435 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 02:29:59.902567 25221 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260619 02:29:59.902675 24495 leader_election.cc:334] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9556c261ac4a46dc97c064bef6d74c73 (127.17.96.254:33963): Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
I20260619 02:29:59.902765 24495 leader_election.cc:302] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 549beb78d8c6431184a1e0282c52e6f0; no voters: 34f02c2e89484735991408738f401ee8, 9556c261ac4a46dc97c064bef6d74c73
I20260619 02:29:59.902971 25220 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 02:30:00.372627 25230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:00.371949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.193:42865)
0619 02:30:00.372361 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:00.372508 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":127,"threads_started":1}
I20260619 02:30:00.377377 25231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:00.376683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.194:48031)
0619 02:30:00.377152 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:00.377245 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":136,"threads_started":1}
I20260619 02:30:00.380131 25232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:00.379516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.195:46691)
0619 02:30:00.379864 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:00.379991 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":105,"threads_started":1}
I20260619 02:30:01.374229 25233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.373581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.193:44145)
0619 02:30:01.373994 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.374091 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":157,"threads_started":1}
I20260619 02:30:01.379153 25234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.378495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.194:34479)
0619 02:30:01.378923 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.379003 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":93,"threads_started":1}
I20260619 02:30:01.381816 25235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.381161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.195:59915)
0619 02:30:01.381541 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.381644 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":113,"threads_started":1}
I20260619 02:30:01.529102 25236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:30:01.529296 25236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: false } }
I20260619 02:30:01.529906 25236 leader_election.cc:288] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9556c261ac4a46dc97c064bef6d74c73 (127.17.96.254:33963), 549beb78d8c6431184a1e0282c52e6f0 (127.17.96.253:32859)
I20260619 02:30:01.530869 25237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.530265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.0.0.1:52324)
0619 02:30:01.530635 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.530741 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":161,"threads_started":1}
I20260619 02:30:01.531244 24518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34f02c2e89484735991408738f401ee8" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "549beb78d8c6431184a1e0282c52e6f0" is_pre_election: true
I20260619 02:30:01.531476 24518 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f02c2e89484735991408738f401ee8 in term 2.
I20260619 02:30:01.531888 24685 leader_election.cc:302] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f02c2e89484735991408738f401ee8, 549beb78d8c6431184a1e0282c52e6f0; no voters:
I20260619 02:30:01.531982 25237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.531649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.0.0.1:52336)
0619 02:30:01.531783 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.531861 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260619 02:30:01.532080 24685 leader_election.cc:334] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9556c261ac4a46dc97c064bef6d74c73 (127.17.96.254:33963): Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
I20260619 02:30:01.532119 25236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 02:30:01.532166 25236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:30:01.532217 25236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 2 FOLLOWER]: Advancing to term 3
I20260619 02:30:01.533440 25236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: false } }
I20260619 02:30:01.533905 25236 leader_election.cc:288] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [CANDIDATE]: Term 3 election: Requested vote from peers 9556c261ac4a46dc97c064bef6d74c73 (127.17.96.254:33963), 549beb78d8c6431184a1e0282c52e6f0 (127.17.96.253:32859)
I20260619 02:30:01.534277 25237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.534029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.0.0.1:52344)
0619 02:30:01.534124 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.534190 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260619 02:30:01.534552 24518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34f02c2e89484735991408738f401ee8" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "549beb78d8c6431184a1e0282c52e6f0"
I20260619 02:30:01.534744 24518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 2 FOLLOWER]: Advancing to term 3
I20260619 02:30:01.535020 25237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.534797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.0.0.1:52354)
0619 02:30:01.534912 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.534966 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260619 02:30:01.535130 24685 leader_election.cc:334] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 9556c261ac4a46dc97c064bef6d74c73 (127.17.96.254:33963): Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
I20260619 02:30:01.536139 24518 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f02c2e89484735991408738f401ee8 in term 3.
I20260619 02:30:01.536598 24685 leader_election.cc:302] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34f02c2e89484735991408738f401ee8, 549beb78d8c6431184a1e0282c52e6f0; no voters: 9556c261ac4a46dc97c064bef6d74c73
I20260619 02:30:01.536887 25236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 02:30:01.537145 25236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [term 3 LEADER]: Becoming Leader. State: Replica: 34f02c2e89484735991408738f401ee8, State: Running, Role: LEADER
I20260619 02:30:01.537460 25236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: false } }
I20260619 02:30:01.538821 25239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34f02c2e89484735991408738f401ee8. Latest consensus state: current_term: 3 leader_uuid: "34f02c2e89484735991408738f401ee8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: false } } }
I20260619 02:30:01.539079 25239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [sys.catalog]: This master's current role is: LEADER
I20260619 02:30:01.539528 25241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:30:01.542135 25241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:30:01.542629 25241 catalog_manager.cc:1259] Loaded cluster ID: 6d424aa9cac949179a6e4c6bf65b8ec0
I20260619 02:30:01.542706 25241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:30:01.543394 25241 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:30:01.543828 25241 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8: Loaded TSK: 0
I20260619 02:30:01.544703 25241 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:30:01.602774 24701 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:30:01.606807 24701 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:30:01.610663 24518 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [term 3 FOLLOWER]: Refusing update from remote peer 34f02c2e89484735991408738f401ee8: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260619 02:30:01.610767 25237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.610370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.0.0.1:52364)
0619 02:30:01.610554 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.610633 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260619 02:30:01.611285 25236 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260619 02:30:01.611596 25237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.611356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.0.0.1:52366)
0619 02:30:01.611480 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.611539 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260619 02:30:01.611730 24685 consensus_peers.cc:487] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 -> Peer 9556c261ac4a46dc97c064bef6d74c73 (127.17.96.254:33963): Couldn't send request to peer 9556c261ac4a46dc97c064bef6d74c73. Status: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 02:30:01.613157 25243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34f02c2e89484735991408738f401ee8. Latest consensus state: current_term: 3 leader_uuid: "34f02c2e89484735991408738f401ee8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: false } } }
I20260619 02:30:01.613329 25243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:30:01.614671 25236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "34f02c2e89484735991408738f401ee8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: false } } }
I20260619 02:30:01.614818 25236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8 [sys.catalog]: This master's current role is: LEADER
I20260619 02:30:01.615163 25243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "34f02c2e89484735991408738f401ee8" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9556c261ac4a46dc97c064bef6d74c73" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 33963 } } peers { permanent_uuid: "549beb78d8c6431184a1e0282c52e6f0" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 32859 } attrs { promote: false } } peers { permanent_uuid: "34f02c2e89484735991408738f401ee8" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 42201 } attrs { promote: false } } }
I20260619 02:30:01.615356 25243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 549beb78d8c6431184a1e0282c52e6f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:30:01.628162 25237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.623220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:37611 (local address 127.0.0.1:43682)
0619 02:30:01.623344 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.623357 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:01.623452 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:01.624168 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:01.624171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:01.624193 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:30:01.624339 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:01.624347 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:01.625533 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:01.625539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:01.626445 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:01.626456 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:01.627665 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:01.627669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:01.627767 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:01.627800 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 02:30:01.627855 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 02:30:01.627957 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"spinlock_wait_cycles":384}
I20260619 02:30:01.629129 25249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.623807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:36321 (local address 127.0.0.1:41336)
0619 02:30:01.624746 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.624761 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:01.624875 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:01.625256 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:01.625259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:01.625279 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:30:01.625365 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:01.625370 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:01.626590 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:01.626594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:01.627622 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:01.627632 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:01.628604 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:01.628609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:01.628703 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:01.628737 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 02:30:01.628846 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 02:30:01.628934 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"thread_start_us":119,"threads_started":1}
I20260619 02:30:01.629323 25250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.623703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:33775 (local address 127.0.0.1:44348)
0619 02:30:01.624073 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.624103 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:01.624183 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:01.624972 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:01.624976 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:01.624994 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:30:01.625216 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:01.625226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:01.626554 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:01.626560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:01.627817 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:01.627827 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:01.628809 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:01.628816 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:01.628924 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:01.628949 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 02:30:01.629048 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 02:30:01.629116 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":57,"thread_start_us":126,"threads_started":1}
I20260619 02:30:01.629348 25251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.623837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44348 (local address 127.17.96.194:33775)
0619 02:30:01.624644 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:01.624652 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:01.624674 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:30:01.624742 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:01.624745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:01.624821 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:01.625347 (+ 526us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:30:01.625507 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:01.626409 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:01.627954 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:01.628616 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:01.628807 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:01.628875 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:01.629067 (+ 192us) server_negotiation.cc:299] Negotiation successful
0619 02:30:01.629160 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":108,"threads_started":1}
I20260619 02:30:01.629791 25248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.623360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43682 (local address 127.17.96.193:37611)
0619 02:30:01.623869 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:01.623874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:01.623890 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:30:01.623950 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:01.623954 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:01.624015 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:01.624460 (+ 445us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:30:01.624572 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:01.625444 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:01.626621 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:01.627350 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:01.629054 (+ 1704us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:01.629133 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:01.629170 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 02:30:01.629622 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":161,"threads_started":1}
I20260619 02:30:01.629805 25252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.624143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41336 (local address 127.17.96.195:36321)
0619 02:30:01.624428 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:01.624435 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:01.624980 (+ 545us) server_negotiation.cc:407] Connection header received
0619 02:30:01.625062 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:01.625066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:01.625126 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:01.625229 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:30:01.625502 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:01.626437 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:01.627792 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:01.628448 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:01.628607 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:01.628689 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:01.629432 (+ 743us) server_negotiation.cc:299] Negotiation successful
0619 02:30:01.629580 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":96,"threads_started":1}
I20260619 02:30:01.630451 24947 tablet_service.cc:1467] Processing CreateTablet for tablet 30d0b79ec0d74f2c93681fd2c9c9ee22 (DEFAULT_TABLE table=test-table [id=8fe159f4b67f4d22a8a4bd79631bd5fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:30:01.630451 25084 tablet_service.cc:1467] Processing CreateTablet for tablet 30d0b79ec0d74f2c93681fd2c9c9ee22 (DEFAULT_TABLE table=test-table [id=8fe159f4b67f4d22a8a4bd79631bd5fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:30:01.630818 24947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30d0b79ec0d74f2c93681fd2c9c9ee22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:01.630836 25084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30d0b79ec0d74f2c93681fd2c9c9ee22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:01.634075 25253 tablet_bootstrap.cc:492] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe: Bootstrap starting.
I20260619 02:30:01.634636 24811 tablet_service.cc:1467] Processing CreateTablet for tablet 30d0b79ec0d74f2c93681fd2c9c9ee22 (DEFAULT_TABLE table=test-table [id=8fe159f4b67f4d22a8a4bd79631bd5fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:30:01.635042 24811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30d0b79ec0d74f2c93681fd2c9c9ee22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:01.636153 25253 tablet_bootstrap.cc:654] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:01.637190 25254 tablet_bootstrap.cc:492] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 2d786c7d81db48f28b4cf8241046399f: Bootstrap starting.
I20260619 02:30:01.638302 25253 tablet_bootstrap.cc:492] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe: No bootstrap required, opened a new log
I20260619 02:30:01.638425 25256 tablet_bootstrap.cc:492] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 153c108b8f4f4fce97375af0dcae499b: Bootstrap starting.
I20260619 02:30:01.638516 25254 tablet_bootstrap.cc:654] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 2d786c7d81db48f28b4cf8241046399f: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:01.638651 25253 ts_tablet_manager.cc:1397] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 02:30:01.639345 25253 raft_consensus.cc:348] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153c108b8f4f4fce97375af0dcae499b" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37611 } } peers { permanent_uuid: "b84f0480fc2c41e691849d8a341d0ffe" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33775 } } peers { permanent_uuid: "2d786c7d81db48f28b4cf8241046399f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36321 } }
I20260619 02:30:01.639528 25253 raft_consensus.cc:374] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:01.639593 25253 raft_consensus.cc:729] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b84f0480fc2c41e691849d8a341d0ffe, State: Initialized, Role: FOLLOWER
I20260619 02:30:01.639691 25256 tablet_bootstrap.cc:654] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 153c108b8f4f4fce97375af0dcae499b: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:01.639819 25253 consensus_queue.cc:260] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153c108b8f4f4fce97375af0dcae499b" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37611 } } peers { permanent_uuid: "b84f0480fc2c41e691849d8a341d0ffe" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33775 } } peers { permanent_uuid: "2d786c7d81db48f28b4cf8241046399f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36321 } }
I20260619 02:30:01.640400 25254 tablet_bootstrap.cc:492] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 2d786c7d81db48f28b4cf8241046399f: No bootstrap required, opened a new log
I20260619 02:30:01.640604 25254 ts_tablet_manager.cc:1397] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 2d786c7d81db48f28b4cf8241046399f: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260619 02:30:01.640690 25253 ts_tablet_manager.cc:1428] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:30:01.641439 25254 raft_consensus.cc:348] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 2d786c7d81db48f28b4cf8241046399f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153c108b8f4f4fce97375af0dcae499b" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37611 } } peers { permanent_uuid: "b84f0480fc2c41e691849d8a341d0ffe" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33775 } } peers { permanent_uuid: "2d786c7d81db48f28b4cf8241046399f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36321 } }
I20260619 02:30:01.641641 25254 raft_consensus.cc:374] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 2d786c7d81db48f28b4cf8241046399f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:01.641687 25254 raft_consensus.cc:729] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 2d786c7d81db48f28b4cf8241046399f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d786c7d81db48f28b4cf8241046399f, State: Initialized, Role: FOLLOWER
I20260619 02:30:01.641876 25234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.641378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.194:32829)
0619 02:30:01.641661 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.641768 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185}
I20260619 02:30:01.641935 25254 consensus_queue.cc:260] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 2d786c7d81db48f28b4cf8241046399f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153c108b8f4f4fce97375af0dcae499b" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37611 } } peers { permanent_uuid: "b84f0480fc2c41e691849d8a341d0ffe" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33775 } } peers { permanent_uuid: "2d786c7d81db48f28b4cf8241046399f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36321 } }
I20260619 02:30:01.642421 25254 ts_tablet_manager.cc:1428] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 2d786c7d81db48f28b4cf8241046399f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:30:01.643276 25234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.642866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.194:33881)
0619 02:30:01.643126 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.643188 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
I20260619 02:30:01.643848 25235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.643493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.195:35671)
0619 02:30:01.643668 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.643732 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260619 02:30:01.644395 25256 tablet_bootstrap.cc:492] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 153c108b8f4f4fce97375af0dcae499b: No bootstrap required, opened a new log
I20260619 02:30:01.644550 25256 ts_tablet_manager.cc:1397] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 153c108b8f4f4fce97375af0dcae499b: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260619 02:30:01.644608 25014 heartbeater.cc:503] Master 127.17.96.252:42201 was elected leader, sending a full tablet report...
I20260619 02:30:01.645269 25235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.644583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.195:42297)
0619 02:30:01.645126 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.645191 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456}
I20260619 02:30:01.645394 25150 heartbeater.cc:503] Master 127.17.96.252:42201 was elected leader, sending a full tablet report...
I20260619 02:30:01.645196 25256 raft_consensus.cc:348] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 153c108b8f4f4fce97375af0dcae499b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153c108b8f4f4fce97375af0dcae499b" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37611 } } peers { permanent_uuid: "b84f0480fc2c41e691849d8a341d0ffe" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33775 } } peers { permanent_uuid: "2d786c7d81db48f28b4cf8241046399f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36321 } }
I20260619 02:30:01.646188 25256 raft_consensus.cc:374] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 153c108b8f4f4fce97375af0dcae499b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:01.646404 25256 raft_consensus.cc:729] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 153c108b8f4f4fce97375af0dcae499b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 153c108b8f4f4fce97375af0dcae499b, State: Initialized, Role: FOLLOWER
I20260619 02:30:01.646731 25256 consensus_queue.cc:260] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 153c108b8f4f4fce97375af0dcae499b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153c108b8f4f4fce97375af0dcae499b" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37611 } } peers { permanent_uuid: "b84f0480fc2c41e691849d8a341d0ffe" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33775 } } peers { permanent_uuid: "2d786c7d81db48f28b4cf8241046399f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36321 } }
I20260619 02:30:01.647362 25256 ts_tablet_manager.cc:1428] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 153c108b8f4f4fce97375af0dcae499b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 02:30:01.648427 25233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.647807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.193:45773)
0619 02:30:01.648269 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.648326 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370}
I20260619 02:30:01.649283 25233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.648993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.193:32867)
0619 02:30:01.649134 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.649194 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260619 02:30:01.649600 24877 heartbeater.cc:503] Master 127.17.96.252:42201 was elected leader, sending a full tablet report...
I20260619 02:30:01.750527 25258 raft_consensus.cc:482] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:30:01.750697 25258 raft_consensus.cc:504] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153c108b8f4f4fce97375af0dcae499b" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37611 } } peers { permanent_uuid: "b84f0480fc2c41e691849d8a341d0ffe" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33775 } } peers { permanent_uuid: "2d786c7d81db48f28b4cf8241046399f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36321 } }
I20260619 02:30:01.752121 25258 leader_election.cc:288] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 153c108b8f4f4fce97375af0dcae499b (127.17.96.193:37611), 2d786c7d81db48f28b4cf8241046399f (127.17.96.195:36321)
I20260619 02:30:01.759135 25234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.752292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:37611 (local address 127.17.96.194:37561)
0619 02:30:01.752678 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.752697 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:01.752820 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:01.753170 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:01.753175 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:01.753197 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:30:01.753418 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:01.753427 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:01.755120 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:01.755125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:01.756958 (+ 1833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:01.756976 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:01.758176 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:01.758181 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:01.758539 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:01.758630 (+ 91us) client_negotiation.cc:769] Sending connection context
0619 02:30:01.758749 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 02:30:01.758902 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":388,"spinlock_wait_cycles":2304}
I20260619 02:30:01.759334 25252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.752566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:52027 (local address 127.17.96.195:36321)
0619 02:30:01.752830 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:01.752835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:01.752850 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:30:01.753012 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:01.753016 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:01.753072 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:01.753154 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:30:01.753554 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:01.754765 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:01.757258 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:01.757998 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:01.758157 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:01.758218 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:01.758992 (+ 774us) server_negotiation.cc:299] Negotiation successful
0619 02:30:01.759144 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260619 02:30:01.759334 25248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.752520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:37561 (local address 127.17.96.193:37611)
0619 02:30:01.752679 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:01.752686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:01.752894 (+ 208us) server_negotiation.cc:407] Connection header received
0619 02:30:01.752960 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:01.752963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:01.753016 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:01.753111 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:30:01.753554 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:01.754765 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:01.757252 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:01.758003 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:01.758158 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:01.758218 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:01.758976 (+ 758us) server_negotiation.cc:299] Negotiation successful
0619 02:30:01.759144 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:30:01.759150 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.752424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:36321 (local address 127.17.96.194:52027)
0619 02:30:01.752774 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.752787 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:01.752865 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:01.753186 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:01.753189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:01.753212 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:30:01.753426 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:01.753432 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:01.755120 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:01.755125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:01.756958 (+ 1833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:01.756976 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:01.758176 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:01.758181 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:01.758521 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:01.758629 (+ 108us) client_negotiation.cc:769] Sending connection context
0619 02:30:01.758749 (+ 120us) client_negotiation.cc:240] Negotiation successful
0619 02:30:01.758906 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":387,"spinlock_wait_cycles":3840,"thread_start_us":136,"threads_started":1}
I20260619 02:30:01.760131 24831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30d0b79ec0d74f2c93681fd2c9c9ee22" candidate_uuid: "b84f0480fc2c41e691849d8a341d0ffe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "153c108b8f4f4fce97375af0dcae499b" is_pre_election: true
I20260619 02:30:01.760404 24831 raft_consensus.cc:2454] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 153c108b8f4f4fce97375af0dcae499b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b84f0480fc2c41e691849d8a341d0ffe in term 0.
I20260619 02:30:01.760586 25104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30d0b79ec0d74f2c93681fd2c9c9ee22" candidate_uuid: "b84f0480fc2c41e691849d8a341d0ffe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d786c7d81db48f28b4cf8241046399f" is_pre_election: true
I20260619 02:30:01.760922 25104 raft_consensus.cc:2454] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 2d786c7d81db48f28b4cf8241046399f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b84f0480fc2c41e691849d8a341d0ffe in term 0.
I20260619 02:30:01.761125 24903 leader_election.cc:302] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 153c108b8f4f4fce97375af0dcae499b, b84f0480fc2c41e691849d8a341d0ffe; no voters:
I20260619 02:30:01.761531 25258 raft_consensus.cc:2781] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:30:01.761637 25258 raft_consensus.cc:482] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:30:01.761704 25258 raft_consensus.cc:3037] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:01.762946 25258 raft_consensus.cc:504] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153c108b8f4f4fce97375af0dcae499b" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37611 } } peers { permanent_uuid: "b84f0480fc2c41e691849d8a341d0ffe" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33775 } } peers { permanent_uuid: "2d786c7d81db48f28b4cf8241046399f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36321 } }
I20260619 02:30:01.763406 25258 leader_election.cc:288] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe [CANDIDATE]: Term 1 election: Requested vote from peers 153c108b8f4f4fce97375af0dcae499b (127.17.96.193:37611), 2d786c7d81db48f28b4cf8241046399f (127.17.96.195:36321)
I20260619 02:30:01.763938 25104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30d0b79ec0d74f2c93681fd2c9c9ee22" candidate_uuid: "b84f0480fc2c41e691849d8a341d0ffe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d786c7d81db48f28b4cf8241046399f"
I20260619 02:30:01.763938 24831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30d0b79ec0d74f2c93681fd2c9c9ee22" candidate_uuid: "b84f0480fc2c41e691849d8a341d0ffe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "153c108b8f4f4fce97375af0dcae499b"
I20260619 02:30:01.764114 24831 raft_consensus.cc:3037] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 153c108b8f4f4fce97375af0dcae499b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:01.764114 25104 raft_consensus.cc:3037] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 2d786c7d81db48f28b4cf8241046399f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:01.765560 24831 raft_consensus.cc:2454] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 153c108b8f4f4fce97375af0dcae499b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b84f0480fc2c41e691849d8a341d0ffe in term 1.
I20260619 02:30:01.765574 25104 raft_consensus.cc:2454] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 2d786c7d81db48f28b4cf8241046399f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b84f0480fc2c41e691849d8a341d0ffe in term 1.
I20260619 02:30:01.766045 24903 leader_election.cc:302] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 153c108b8f4f4fce97375af0dcae499b, b84f0480fc2c41e691849d8a341d0ffe; no voters:
I20260619 02:30:01.766338 25258 raft_consensus.cc:2781] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:30:01.766644 25258 raft_consensus.cc:686] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe [term 1 LEADER]: Becoming Leader. State: Replica: b84f0480fc2c41e691849d8a341d0ffe, State: Running, Role: LEADER
I20260619 02:30:01.766968 25258 consensus_queue.cc:237] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153c108b8f4f4fce97375af0dcae499b" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37611 } } peers { permanent_uuid: "b84f0480fc2c41e691849d8a341d0ffe" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33775 } } peers { permanent_uuid: "2d786c7d81db48f28b4cf8241046399f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36321 } }
I20260619 02:30:01.767859 25234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.767272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.194:33769)
0619 02:30:01.767683 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.767763 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260619 02:30:01.770812 24701 catalog_manager.cc:5495] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe reported cstate change: term changed from 0 to 1, leader changed from <none> to b84f0480fc2c41e691849d8a341d0ffe (127.17.96.194). New cstate: current_term: 1 leader_uuid: "b84f0480fc2c41e691849d8a341d0ffe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "153c108b8f4f4fce97375af0dcae499b" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b84f0480fc2c41e691849d8a341d0ffe" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d786c7d81db48f28b4cf8241046399f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36321 } health_report { overall_health: UNKNOWN } } }
I20260619 02:30:01.797408 25251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.790085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44364 (local address 127.17.96.194:33775)
0619 02:30:01.790240 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:01.790247 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:01.790347 (+ 100us) server_negotiation.cc:407] Connection header received
0619 02:30:01.790604 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:01.790608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:01.790675 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:01.790816 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:30:01.791134 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:01.791927 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:01.792952 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:01.793195 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:01.793314 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:01.794018 (+ 704us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:30:01.795235 (+ 1217us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:30:01.795562 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:01.797037 (+ 1475us) server_negotiation.cc:299] Negotiation successful
0619 02:30:01.797156 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:30:01.802065 24831 raft_consensus.cc:1261] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 153c108b8f4f4fce97375af0dcae499b [term 1 FOLLOWER]: Refusing update from remote peer b84f0480fc2c41e691849d8a341d0ffe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:30:01.802973 25104 raft_consensus.cc:1261] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 2d786c7d81db48f28b4cf8241046399f [term 1 FOLLOWER]: Refusing update from remote peer b84f0480fc2c41e691849d8a341d0ffe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:30:01.803755 25263 consensus_queue.cc:1035] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe [LEADER]: Connected to new peer: Peer: permanent_uuid: "153c108b8f4f4fce97375af0dcae499b" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:30:01.804033 25258 consensus_queue.cc:1035] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d786c7d81db48f28b4cf8241046399f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:30:01.806178 25233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.805819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.193:34371)
0619 02:30:01.805965 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.806051 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260619 02:30:01.807497 25235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.806906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.195:57751)
0619 02:30:01.807300 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.807381 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305}
I20260619 02:30:01.810076 25234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.809450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.194:43983)
0619 02:30:01.809879 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.809959 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335}
I20260619 02:30:01.810681 25233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.810389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.193:50585)
0619 02:30:01.810536 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.810599 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260619 02:30:01.811707 25234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.811430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.194:50669)
0619 02:30:01.811562 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.811629 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260619 02:30:01.812810 25235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.812384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:33963 (local address 127.17.96.195:58501)
0619 02:30:01.812633 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.812707 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
I20260619 02:30:01.814232 25268 mvcc.cc:205] Tried to move back new op lower bound from 7298401082572673024 to 7298401082442915840. Current Snapshot: MvccSnapshot[applied={T|T < 7298401082572673024}]
I20260619 02:30:01.824826 24701 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:30:01.825068 24701 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:30:01.830358 24701 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34f02c2e89484735991408738f401ee8: Sending DeleteTablet for 3 replicas of tablet 30d0b79ec0d74f2c93681fd2c9c9ee22
I20260619 02:30:01.831176 24811 tablet_service.cc:1514] Processing DeleteTablet for tablet 30d0b79ec0d74f2c93681fd2c9c9ee22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:30:01 UTC) from {username='oryx', principal='oryx/127.17.96.252@KRBTEST.COM'} at 127.0.0.1:43682
I20260619 02:30:01.831197 24947 tablet_service.cc:1514] Processing DeleteTablet for tablet 30d0b79ec0d74f2c93681fd2c9c9ee22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:30:01 UTC) from {username='oryx', principal='oryx/127.17.96.252@KRBTEST.COM'} at 127.0.0.1:44348
I20260619 02:30:01.831652 25277 tablet_replica.cc:330] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 153c108b8f4f4fce97375af0dcae499b: stopping tablet replica
I20260619 02:30:01.831795 25277 raft_consensus.cc:2229] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 153c108b8f4f4fce97375af0dcae499b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:30:01.831843 25084 tablet_service.cc:1514] Processing DeleteTablet for tablet 30d0b79ec0d74f2c93681fd2c9c9ee22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:30:01 UTC) from {username='oryx', principal='oryx/127.17.96.252@KRBTEST.COM'} at 127.0.0.1:41336
I20260619 02:30:01.831961 25277 raft_consensus.cc:2258] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 153c108b8f4f4fce97375af0dcae499b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:30:01.832258 25276 tablet_replica.cc:330] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe: stopping tablet replica
I20260619 02:30:01.832401 25276 raft_consensus.cc:2229] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:30:01.832481 25278 tablet_replica.cc:330] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 2d786c7d81db48f28b4cf8241046399f: stopping tablet replica
I20260619 02:30:01.832603 25278 raft_consensus.cc:2229] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 2d786c7d81db48f28b4cf8241046399f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:30:01.832755 25278 raft_consensus.cc:2258] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 2d786c7d81db48f28b4cf8241046399f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:30:01.832780 25276 raft_consensus.cc:2258] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:30:01.833105 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 24652
I20260619 02:30:01.834270 25276 ts_tablet_manager.cc:1905] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:30:01.835474 25278 ts_tablet_manager.cc:1905] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 2d786c7d81db48f28b4cf8241046399f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:30:01.839160 25276 ts_tablet_manager.cc:1918] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:30:01.839282 25276 log.cc:1196] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/30d0b79ec0d74f2c93681fd2c9c9ee22
I20260619 02:30:01.839540 25278 ts_tablet_manager.cc:1918] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 2d786c7d81db48f28b4cf8241046399f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:30:01.839643 25278 log.cc:1196] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 2d786c7d81db48f28b4cf8241046399f: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/30d0b79ec0d74f2c93681fd2c9c9ee22
I20260619 02:30:01.839682 25276 ts_tablet_manager.cc:1939] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P b84f0480fc2c41e691849d8a341d0ffe: Deleting consensus metadata
I20260619 02:30:01.839967 25278 ts_tablet_manager.cc:1939] T 30d0b79ec0d74f2c93681fd2c9c9ee22 P 2d786c7d81db48f28b4cf8241046399f: Deleting consensus metadata
I20260619 02:30:01.840848 24683 catalog_manager.cc:4841] TS b84f0480fc2c41e691849d8a341d0ffe (127.17.96.194:33775): tablet 30d0b79ec0d74f2c93681fd2c9c9ee22 (table test-table [id=8fe159f4b67f4d22a8a4bd79631bd5fd]) successfully deleted
I20260619 02:30:01.841321 24682 catalog_manager.cc:4841] TS 2d786c7d81db48f28b4cf8241046399f (127.17.96.195:36321): tablet 30d0b79ec0d74f2c93681fd2c9c9ee22 (table test-table [id=8fe159f4b67f4d22a8a4bd79631bd5fd]) successfully deleted
I20260619 02:30:01.846437 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 24885
I20260619 02:30:01.846725 25249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:01.846307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:37611 (local address 127.0.0.1:43692)
0619 02:30:01.846519 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:01.846591 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.193:37611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260619 02:30:01.848269 24684 catalog_manager.cc:4584] TS 153c108b8f4f4fce97375af0dcae499b (127.17.96.193:37611): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 30d0b79ec0d74f2c93681fd2c9c9ee22: Network error: Client connection negotiation failed: client connection to 127.17.96.193:37611: connect: Connection refused (error 111)
I20260619 02:30:01.854554 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 25021
I20260619 02:30:01.862648 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 24371
I20260619 02:30:01.870107 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 24472
2026-06-19T02:30:01Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7757 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:30:01 dist-test-slave-4qw6 krb5kdc[25281](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:30:01 dist-test-slave-4qw6 krb5kdc[25281](info): set up 2 sockets
Jun 19 02:30:01 dist-test-slave-4qw6 krb5kdc[25281](info): commencing operation
krb5kdc: starting...
W20260619 02:30:03.026978 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:30:03 dist-test-slave-4qw6 krb5kdc[25281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:30:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:30:03Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.254.keytab.
Entry for principal oryx/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.254.keytab.
I20260619 02:30:03.178435 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:37977
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.254.keytab
--principal=oryx/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:41947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:37977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:30:03.311297 25294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:03.311666 25294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:03.311719 25294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:03.316385 25294 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:30:03.316481 25294 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 02:30:03.316505 25294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:03.316529 25294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:03.316550 25294 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:30:03.316570 25294 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:30:03.322907 25294 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:41947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:37977
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.254.keytab
--principal=oryx/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:37977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:03.324200 25294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:03.325342 25294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:03.335955 25299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:03.335958 25302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:03.335994 25300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:03.336068 25294 server_base.cc:1034] running on GCE node
I20260619 02:30:03.336879 25294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:03.338258 25294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:03.339511 25294 hybrid_clock.cc:648] HybridClock initialized: now 1781836203339498 us; error 40 us; skew 500 ppm
Jun 19 02:30:03 dist-test-slave-4qw6 krb5kdc[25281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836203, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:30:03.342994 25294 init.cc:377] Logged in from keytab as oryx/127.17.96.254@KRBTEST.COM (short username oryx)
I20260619 02:30:03.344224 25294 webserver.cc:460] Webserver started at http://127.17.96.254:37955/ using document root <none> and password file <none>
I20260619 02:30:03.344931 25294 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:03.345059 25294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:03.345341 25294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:30:03.348011 25294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "dc018e2bb5d0430ea20129395ff03272"
format_stamp: "Formatted at 2026-06-19 02:30:03 on dist-test-slave-4qw6"
server_key: "9c56d7a2bf193293f3de230c9591acdd"
server_key_iv: "163aa205fbc16075fa32513a279a6494"
server_key_version: "encryptionkey@0"
I20260619 02:30:03.348754 25294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "dc018e2bb5d0430ea20129395ff03272"
format_stamp: "Formatted at 2026-06-19 02:30:03 on dist-test-slave-4qw6"
server_key: "9c56d7a2bf193293f3de230c9591acdd"
server_key_iv: "163aa205fbc16075fa32513a279a6494"
server_key_version: "encryptionkey@0"
I20260619 02:30:03.353458 25294 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260619 02:30:03.356971 25309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:03.358634 25294 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:30:03.358842 25294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "dc018e2bb5d0430ea20129395ff03272"
format_stamp: "Formatted at 2026-06-19 02:30:03 on dist-test-slave-4qw6"
server_key: "9c56d7a2bf193293f3de230c9591acdd"
server_key_iv: "163aa205fbc16075fa32513a279a6494"
server_key_version: "encryptionkey@0"
I20260619 02:30:03.359011 25294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:03.389395 25294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:03.393524 25294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:03.393833 25294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:03.406028 25294 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:37977
I20260619 02:30:03.406075 25371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:37977 every 8 connection(s)
I20260619 02:30:03.407547 25294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:30:03.411335 25372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:03.417904 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 25294
I20260619 02:30:03.418079 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:30:03.418431 17795 external_mini_cluster.cc:1428] Setting key b67cfd88953318b9d9f40926bfbb86f7
I20260619 02:30:03.418813 25372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272: Bootstrap starting.
I20260619 02:30:03.422417 25372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:03.423525 25372 log.cc:824] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272: Log is configured to *not* fsync() on all Append() calls
I20260619 02:30:03.426376 25372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272: No bootstrap required, opened a new log
I20260619 02:30:03.430683 25372 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc018e2bb5d0430ea20129395ff03272" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 37977 } }
I20260619 02:30:03.431058 25372 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:03.431155 25372 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc018e2bb5d0430ea20129395ff03272, State: Initialized, Role: FOLLOWER
I20260619 02:30:03.431766 25372 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc018e2bb5d0430ea20129395ff03272" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 37977 } }
I20260619 02:30:03.431983 25372 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:30:03.432057 25372 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:30:03.432175 25372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:03.433589 25372 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc018e2bb5d0430ea20129395ff03272" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 37977 } }
Jun 19 02:30:03 dist-test-slave-4qw6 krb5kdc[25281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM
I20260619 02:30:03.434073 25372 leader_election.cc:302] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc018e2bb5d0430ea20129395ff03272; no voters:
I20260619 02:30:03.434460 25372 leader_election.cc:288] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:30:03.434653 25377 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:30:03.434971 25377 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272 [term 1 LEADER]: Becoming Leader. State: Replica: dc018e2bb5d0430ea20129395ff03272, State: Running, Role: LEADER
I20260619 02:30:03.435376 25377 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc018e2bb5d0430ea20129395ff03272" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 37977 } }
I20260619 02:30:03.436160 25372 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:30:03.437460 25378 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc018e2bb5d0430ea20129395ff03272" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc018e2bb5d0430ea20129395ff03272" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 37977 } } }
I20260619 02:30:03.437620 25378 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272 [sys.catalog]: This master's current role is: LEADER
I20260619 02:30:03.438135 25379 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc018e2bb5d0430ea20129395ff03272. Latest consensus state: current_term: 1 leader_uuid: "dc018e2bb5d0430ea20129395ff03272" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc018e2bb5d0430ea20129395ff03272" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 37977 } } }
I20260619 02:30:03.438458 25379 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272 [sys.catalog]: This master's current role is: LEADER
I20260619 02:30:03.441637 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:03.420405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41348 (local address 127.17.96.254:37977)
0619 02:30:03.421096 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:03.421124 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:03.421185 (+ 61us) server_negotiation.cc:407] Connection header received
0619 02:30:03.422258 (+ 1073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:03.422289 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:03.422923 (+ 634us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:03.423377 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:03.424680 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:03.425968 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:03.427096 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:03.427690 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:03.427946 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:03.434565 (+ 6619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:03.434575 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:03.434600 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:30:03.434655 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:03.437410 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:03.438313 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:03.438320 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:03.438328 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:03.438528 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:03.438964 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:03.438968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:03.438970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:03.439698 (+ 728us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:03.439972 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:03.440304 (+ 332us) server_negotiation.cc:299] Negotiation successful
0619 02:30:03.440733 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":218,"threads_started":1}
W20260619 02:30:03.443070 25392 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:30:03.443187 25392 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:30:03.443387 25386 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:30:03.444429 25386 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:30:03.451398 25386 catalog_manager.cc:1347] Generated new cluster ID: 42790ba6dcdb46408f391c2fa06310b9
I20260619 02:30:03.451514 25386 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:30:03.464305 25386 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:30:03.465916 25386 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:30:03.474596 25386 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272: Generated new TSK 0
I20260619 02:30:03.475669 25386 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.96.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.96.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.96.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.193.keytab.
Entry for principal oryx/127.17.96.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.193.keytab.
I20260619 02:30:03.518296 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:37977
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.193.keytab
--principal=oryx/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:41947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:30:03.659675 25398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:03.660043 25398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:03.660100 25398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:03.665337 25398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:03.665449 25398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:03.665531 25398 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:30:03.665621 25398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:30:03.672103 25398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:41947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.193.keytab
--principal=oryx/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.96.254:37977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:03.673408 25398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:03.674600 25398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:03.686825 25404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:03.686830 25403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:03.687201 25398 server_base.cc:1034] running on GCE node
W20260619 02:30:03.686852 25406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:03.687896 25398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:03.688683 25398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:03.689859 25398 hybrid_clock.cc:648] HybridClock initialized: now 1781836203689843 us; error 45 us; skew 500 ppm
Jun 19 02:30:03 dist-test-slave-4qw6 krb5kdc[25281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836203, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:30:03.693323 25398 init.cc:377] Logged in from keytab as oryx/127.17.96.193@KRBTEST.COM (short username oryx)
I20260619 02:30:03.694523 25398 webserver.cc:460] Webserver started at http://127.17.96.193:43283/ using document root <none> and password file <none>
I20260619 02:30:03.695247 25398 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:03.695334 25398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:03.695557 25398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:30:03.698179 25398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "d1d857a119474703901ea085386d00c5"
format_stamp: "Formatted at 2026-06-19 02:30:03 on dist-test-slave-4qw6"
server_key: "5c2c48c5bbe4ef37377f8bca4e091271"
server_key_iv: "56f4df612074455d076e4e57bed25721"
server_key_version: "encryptionkey@0"
I20260619 02:30:03.698861 25398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "d1d857a119474703901ea085386d00c5"
format_stamp: "Formatted at 2026-06-19 02:30:03 on dist-test-slave-4qw6"
server_key: "5c2c48c5bbe4ef37377f8bca4e091271"
server_key_iv: "56f4df612074455d076e4e57bed25721"
server_key_version: "encryptionkey@0"
I20260619 02:30:03.703755 25398 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:30:03.707188 25413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:03.708833 25398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:30:03.709060 25398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "d1d857a119474703901ea085386d00c5"
format_stamp: "Formatted at 2026-06-19 02:30:03 on dist-test-slave-4qw6"
server_key: "5c2c48c5bbe4ef37377f8bca4e091271"
server_key_iv: "56f4df612074455d076e4e57bed25721"
server_key_version: "encryptionkey@0"
I20260619 02:30:03.709228 25398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:03.726871 25398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:03.730756 25398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:03.731009 25398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:03.732776 25398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:30:03.732864 25398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:03.732944 25398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:30:03.733006 25398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:30:03 dist-test-slave-4qw6 krb5kdc[25281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836203, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM
I20260619 02:30:03.746601 25398 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:37233
I20260619 02:30:03.746660 25528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:37233 every 8 connection(s)
I20260619 02:30:03.748018 25398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:30:03.748535 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 25398
I20260619 02:30:03.748708 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:30:03.749070 17795 external_mini_cluster.cc:1428] Setting key 760662ef91cec51d1d55a1e06423385b
I20260619 02:30:03.750671 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:03.733734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:56563 (local address 127.17.96.254:37977)
0619 02:30:03.733897 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:03.733900 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:03.735100 (+ 1200us) server_negotiation.cc:407] Connection header received
0619 02:30:03.736387 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:03.736391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:03.736466 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:03.736517 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:03.738718 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:03.739451 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:03.740855 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:03.741150 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:03.741289 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:03.745575 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:03.745579 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:03.745583 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:30:03.745624 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:03.747682 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:03.748401 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:03.748408 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:03.748414 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:03.748548 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:03.749098 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:03.749101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:03.749103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:03.749455 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:03.749579 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:03.750298 (+ 719us) server_negotiation.cc:299] Negotiation successful
0619 02:30:03.750464 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 02:30:03.751634 25422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:03.734384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:37977 (local address 127.17.96.193:56563)
0619 02:30:03.734921 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:03.734972 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:03.736093 (+ 1121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:03.737053 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:03.737063 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:03.737657 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:03.738523 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:03.738541 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:03.739623 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:03.739627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:03.740692 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:03.740704 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:03.741353 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:03.741358 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:03.741684 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:03.742657 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:03.742683 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:03.745334 (+ 2651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:30:03.747844 (+ 2510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:03.747850 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:03.747863 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:03.748239 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:03.748730 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:03.748733 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:03.748736 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:03.748942 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:03.749581 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:30:03.749587 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:30:03.750003 (+ 416us) client_negotiation.cc:769] Sending connection context
0619 02:30:03.750226 (+ 223us) client_negotiation.cc:240] Negotiation successful
0619 02:30:03.750626 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":136,"threads_started":1}
I20260619 02:30:03.753041 25529 heartbeater.cc:347] Connected to a master server at 127.17.96.254:37977
I20260619 02:30:03.753242 25529 heartbeater.cc:464] Registering TS with master...
I20260619 02:30:03.753883 25529 heartbeater.cc:511] Master 127.17.96.254:37977 requested a full tablet report, sending...
I20260619 02:30:03.755571 25325 ts_manager.cc:194] Registered new tserver with Master: d1d857a119474703901ea085386d00c5 (127.17.96.193:37233)
I20260619 02:30:03.757325 25325 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:56563
WARNING: no policy specified for oryx/127.17.96.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.96.194@KRBTEST.COM" created.
I20260619 02:30:03.765122 25536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:03.756180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:37977 (local address 127.17.96.193:56671)
0619 02:30:03.756559 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:03.756577 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:03.756699 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:03.757014 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:03.757017 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:03.757266 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:03.757411 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:03.757421 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:03.758542 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:03.758545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:03.759180 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:03.759189 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:03.759751 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:03.759755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:03.759860 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:03.760514 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:03.760539 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:03.760920 (+ 381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:30:03.763164 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:03.763168 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:03.763171 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:03.763491 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:03.763839 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:03.763844 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:03.763847 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:03.763949 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:03.764516 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:30:03.764523 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:30:03.764648 (+ 125us) client_negotiation.cc:769] Sending connection context
0619 02:30:03.764765 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 02:30:03.764929 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":159,"threads_started":1}
I20260619 02:30:03.765309 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:03.756078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:56671 (local address 127.17.96.254:37977)
0619 02:30:03.756228 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:03.756233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:03.756660 (+ 427us) server_negotiation.cc:407] Connection header received
0619 02:30:03.756797 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:03.756804 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:03.756873 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:03.756981 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:03.757571 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:03.758363 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:03.759323 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:03.759610 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:03.759743 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:03.761105 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:03.761109 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:03.761113 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:30:03.761144 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:03.763020 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:03.763608 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:03.763613 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:03.763615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:03.763719 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:03.764065 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:03.764070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:03.764072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:03.764354 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:03.764531 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:03.764861 (+ 330us) server_negotiation.cc:299] Negotiation successful
0619 02:30:03.765025 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.96.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.194.keytab.
Entry for principal oryx/127.17.96.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.194.keytab.
I20260619 02:30:03.781206 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:37977
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.194.keytab
--principal=oryx/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:41947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:30:03.911257 25538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:03.911587 25538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:03.911638 25538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:03.916416 25538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:03.916517 25538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:03.916563 25538 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:30:03.916600 25538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:30:03.923199 25538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:41947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.194.keytab
--principal=oryx/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.96.254:37977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:03.924587 25538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:03.925817 25538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:03.937752 25543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:03.937969 25538 server_base.cc:1034] running on GCE node
W20260619 02:30:03.937776 25546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:03.937776 25544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:03.938947 25538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:03.939857 25538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:03.941062 25538 hybrid_clock.cc:648] HybridClock initialized: now 1781836203941012 us; error 71 us; skew 500 ppm
Jun 19 02:30:03 dist-test-slave-4qw6 krb5kdc[25281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836203, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:30:03.944410 25538 init.cc:377] Logged in from keytab as oryx/127.17.96.194@KRBTEST.COM (short username oryx)
I20260619 02:30:03.945638 25538 webserver.cc:460] Webserver started at http://127.17.96.194:33509/ using document root <none> and password file <none>
I20260619 02:30:03.946336 25538 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:03.946412 25538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:03.946614 25538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:30:03.948952 25538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "393b9d3c902f4989b620d5db1cc80e01"
format_stamp: "Formatted at 2026-06-19 02:30:03 on dist-test-slave-4qw6"
server_key: "f9536cec680fc1a92e75dc5caae7bfb0"
server_key_iv: "ee3059b8ea01e736a2b4f6b00676d10a"
server_key_version: "encryptionkey@0"
I20260619 02:30:03.949576 25538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "393b9d3c902f4989b620d5db1cc80e01"
format_stamp: "Formatted at 2026-06-19 02:30:03 on dist-test-slave-4qw6"
server_key: "f9536cec680fc1a92e75dc5caae7bfb0"
server_key_iv: "ee3059b8ea01e736a2b4f6b00676d10a"
server_key_version: "encryptionkey@0"
I20260619 02:30:03.954241 25538 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 02:30:03.957630 25553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:03.959244 25538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:30:03.959456 25538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "393b9d3c902f4989b620d5db1cc80e01"
format_stamp: "Formatted at 2026-06-19 02:30:03 on dist-test-slave-4qw6"
server_key: "f9536cec680fc1a92e75dc5caae7bfb0"
server_key_iv: "ee3059b8ea01e736a2b4f6b00676d10a"
server_key_version: "encryptionkey@0"
I20260619 02:30:03.959623 25538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:03.976217 25538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:03.980619 25538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:03.980957 25538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:03.983057 25538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:30:03.983137 25538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:03.983213 25538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:30:03.983258 25538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:30:03 dist-test-slave-4qw6 krb5kdc[25281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836203, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM
I20260619 02:30:03.996420 25538 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:42291
I20260619 02:30:03.996511 25668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:42291 every 8 connection(s)
I20260619 02:30:03.997627 25538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:30:04.000252 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 25538
I20260619 02:30:04.000308 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:03.984054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:48105 (local address 127.17.96.254:37977)
0619 02:30:03.984304 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:03.984313 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:03.985333 (+ 1020us) server_negotiation.cc:407] Connection header received
0619 02:30:03.986629 (+ 1296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:03.986638 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:03.986722 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:03.986854 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:03.988513 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:03.989531 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:03.991083 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:03.991332 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:03.991468 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:03.995094 (+ 3626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:03.995099 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:03.995103 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:30:03.995151 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:03.997204 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:03.998084 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:03.998088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:03.998090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:03.998225 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:03.998794 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:03.998796 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:03.998798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:03.999034 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:03.999143 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:03.999925 (+ 782us) server_negotiation.cc:299] Negotiation successful
0619 02:30:04.000104 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260619 02:30:04.000406 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:30:04.000758 17795 external_mini_cluster.cc:1428] Setting key d37946c64225eb83045ff67680cd959a
I20260619 02:30:04.001314 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:03.984501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:37977 (local address 127.17.96.194:48105)
0619 02:30:03.985124 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:03.985186 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:03.986288 (+ 1102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:03.987189 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:03.987198 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:03.987733 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:03.988306 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:03.988323 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:03.989738 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:03.989766 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:03.990917 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:03.990928 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:03.991539 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:03.991544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:03.991744 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:03.992479 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:03.992497 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:03.994749 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:30:03.997397 (+ 2648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:03.997404 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:03.997420 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:03.997896 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:03.998413 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:03.998417 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:03.998419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:03.998646 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:03.999165 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:30:03.999171 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:30:03.999476 (+ 305us) client_negotiation.cc:769] Sending connection context
0619 02:30:03.999662 (+ 186us) client_negotiation.cc:240] Negotiation successful
0619 02:30:04.000070 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":178,"threads_started":1}
I20260619 02:30:04.002825 25669 heartbeater.cc:347] Connected to a master server at 127.17.96.254:37977
I20260619 02:30:04.003017 25669 heartbeater.cc:464] Registering TS with master...
I20260619 02:30:04.003685 25669 heartbeater.cc:511] Master 127.17.96.254:37977 requested a full tablet report, sending...
I20260619 02:30:04.005007 25324 ts_manager.cc:194] Registered new tserver with Master: 393b9d3c902f4989b620d5db1cc80e01 (127.17.96.194:42291)
I20260619 02:30:04.005959 25324 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:48105
WARNING: no policy specified for oryx/127.17.96.195@KRBTEST.COM; defaulting to no policy
I20260619 02:30:04.015668 25676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.005189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:37977 (local address 127.17.96.194:46607)
0619 02:30:04.005872 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:04.005888 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:04.006007 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:04.006496 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:04.006502 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:04.006784 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:04.006977 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.006989 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.008095 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.008101 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.008914 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.008925 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.009685 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.009688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.009828 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.010495 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:04.010507 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:04.010892 (+ 385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:30:04.013154 (+ 2262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:04.013157 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:04.013159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:04.013388 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:04.013715 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:04.013718 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:04.013719 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:04.013878 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:04.014878 (+ 1000us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:30:04.014881 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:30:04.015000 (+ 119us) client_negotiation.cc:769] Sending connection context
0619 02:30:04.015104 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 02:30:04.015490 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":112,"threads_started":1}
I20260619 02:30:04.015682 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.005545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:46607 (local address 127.17.96.254:37977)
0619 02:30:04.005669 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:04.005674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:04.005989 (+ 315us) server_negotiation.cc:407] Connection header received
0619 02:30:04.006167 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:04.006174 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:04.006240 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:04.006343 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:04.007139 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.007932 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.009064 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.009525 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.009697 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.011087 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:04.011095 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:04.011100 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:30:04.011135 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:04.013010 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:04.013519 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:04.013522 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:04.013523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:04.013618 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:04.014004 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:04.014006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:04.014007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:04.014662 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:04.014876 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:04.015322 (+ 446us) server_negotiation.cc:299] Negotiation successful
0619 02:30:04.015508 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.96.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.96.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.195.keytab.
Entry for principal oryx/127.17.96.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.195.keytab.
I20260619 02:30:04.039801 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:37977
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.195.keytab
--principal=oryx/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:41947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:30:04.174111 25678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:04.174455 25678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:04.174549 25678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:04.179352 25678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:04.179442 25678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:04.179489 25678 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:30:04.179527 25678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:30:04.185801 25678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:41947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.195.keytab
--principal=oryx/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.96.254:37977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:04.187175 25678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:04.188455 25678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:04.200376 25684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:04.200353 25683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:04.200517 25686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:04.200680 25678 server_base.cc:1034] running on GCE node
I20260619 02:30:04.201851 25678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:04.203151 25678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:04.206437 25678 hybrid_clock.cc:648] HybridClock initialized: now 1781836204206347 us; error 118 us; skew 500 ppm
Jun 19 02:30:04 dist-test-slave-4qw6 krb5kdc[25281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836204, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:30:04.210088 25678 init.cc:377] Logged in from keytab as oryx/127.17.96.195@KRBTEST.COM (short username oryx)
I20260619 02:30:04.211202 25678 webserver.cc:460] Webserver started at http://127.17.96.195:42709/ using document root <none> and password file <none>
I20260619 02:30:04.211849 25678 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:04.211927 25678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:04.212128 25678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:30:04.214685 25678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "6fce9948443c4324a28308ed2586472a"
format_stamp: "Formatted at 2026-06-19 02:30:04 on dist-test-slave-4qw6"
server_key: "5f4d8b41c550c76d83825961faae40b7"
server_key_iv: "1138ecb6b92ccc2fa0bc1c4a66163375"
server_key_version: "encryptionkey@0"
I20260619 02:30:04.215390 25678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "6fce9948443c4324a28308ed2586472a"
format_stamp: "Formatted at 2026-06-19 02:30:04 on dist-test-slave-4qw6"
server_key: "5f4d8b41c550c76d83825961faae40b7"
server_key_iv: "1138ecb6b92ccc2fa0bc1c4a66163375"
server_key_version: "encryptionkey@0"
I20260619 02:30:04.221920 25678 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260619 02:30:04.225427 25693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:04.227201 25678 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 02:30:04.227344 25678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "6fce9948443c4324a28308ed2586472a"
format_stamp: "Formatted at 2026-06-19 02:30:04 on dist-test-slave-4qw6"
server_key: "5f4d8b41c550c76d83825961faae40b7"
server_key_iv: "1138ecb6b92ccc2fa0bc1c4a66163375"
server_key_version: "encryptionkey@0"
I20260619 02:30:04.227530 25678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:04.248205 25678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:04.252457 25678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:04.252733 25678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:04.255609 25678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:30:04.255738 25678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:04.255812 25678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:30:04.255848 25678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:30:04 dist-test-slave-4qw6 krb5kdc[25281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836204, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM
I20260619 02:30:04.270259 25678 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:42093
I20260619 02:30:04.270329 25808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:42093 every 8 connection(s)
I20260619 02:30:04.271538 25678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:30:04.274596 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.256906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:51045 (local address 127.17.96.254:37977)
0619 02:30:04.257150 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:04.257157 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:04.258268 (+ 1111us) server_negotiation.cc:407] Connection header received
0619 02:30:04.259628 (+ 1360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:04.259635 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:04.259722 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:04.259826 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:04.262036 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.262690 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.264731 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.265188 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.265355 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.269691 (+ 4336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:04.269697 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:04.269701 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:30:04.269741 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:04.271548 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:04.272257 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:04.272261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:04.272264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:04.272366 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:04.272928 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:04.272934 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:04.272936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:04.273289 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:04.273413 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:04.274225 (+ 812us) server_negotiation.cc:299] Negotiation successful
0619 02:30:04.274421 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 02:30:04.275384 25702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.257450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:37977 (local address 127.17.96.195:51045)
0619 02:30:04.258052 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:04.258113 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:04.259272 (+ 1159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:04.260354 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:04.260365 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:04.260932 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:04.261820 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.261851 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.262845 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.262849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.264577 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.264591 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.265476 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.265486 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.265866 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.266855 (+ 989us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:04.266888 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:04.269407 (+ 2519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:30:04.271715 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:04.271722 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:04.271740 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:04.272077 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:04.272484 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:04.272487 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:04.272490 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:04.272718 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:04.273435 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:30:04.273442 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:30:04.273740 (+ 298us) client_negotiation.cc:769] Sending connection context
0619 02:30:04.274025 (+ 285us) client_negotiation.cc:240] Negotiation successful
0619 02:30:04.274352 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":173,"threads_started":1}
I20260619 02:30:04.277532 25809 heartbeater.cc:347] Connected to a master server at 127.17.96.254:37977
I20260619 02:30:04.277845 25809 heartbeater.cc:464] Registering TS with master...
I20260619 02:30:04.278719 25809 heartbeater.cc:511] Master 127.17.96.254:37977 requested a full tablet report, sending...
I20260619 02:30:04.279798 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 25678
I20260619 02:30:04.279978 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:30:04.280221 25324 ts_manager.cc:194] Registered new tserver with Master: 6fce9948443c4324a28308ed2586472a (127.17.96.195:42093)
I20260619 02:30:04.280334 17795 external_mini_cluster.cc:1428] Setting key 7567a16bef7aed47a9a8734bd0846a9d
I20260619 02:30:04.281190 25324 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:51045
I20260619 02:30:04.283164 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:30:04.289677 25815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.280490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:37977 (local address 127.17.96.195:35387)
0619 02:30:04.281119 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:04.281136 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:04.281283 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:04.281601 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:04.281604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:04.281923 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:04.282110 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.282120 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.283158 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.283162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.283892 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.283903 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.284434 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.284438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.284538 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.285196 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:04.285220 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:04.285836 (+ 616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:30:04.287788 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:04.287795 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:04.287800 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:04.288119 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:04.288477 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:04.288483 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:04.288487 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:04.288627 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:04.289059 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:30:04.289063 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:30:04.289195 (+ 132us) client_negotiation.cc:769] Sending connection context
0619 02:30:04.289329 (+ 134us) client_negotiation.cc:240] Negotiation successful
0619 02:30:04.289502 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":171,"threads_started":1}
I20260619 02:30:04.289816 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.280798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:35387 (local address 127.17.96.254:37977)
0619 02:30:04.280976 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:04.280982 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:04.281352 (+ 370us) server_negotiation.cc:407] Connection header received
0619 02:30:04.281401 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:04.281404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:04.281475 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:04.281551 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:04.282295 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.282975 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.284051 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.284297 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.284425 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.286042 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:04.286045 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:04.286048 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:30:04.286076 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:04.287648 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:04.288250 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:04.288254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:04.288256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:04.288361 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:04.288731 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:04.288733 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:04.288734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:04.288951 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:04.289039 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:04.289413 (+ 374us) server_negotiation.cc:299] Negotiation successful
0619 02:30:04.289567 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
Jun 19 02:30:04 dist-test-slave-4qw6 krb5kdc[25281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836204, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 02:30:04 dist-test-slave-4qw6 krb5kdc[25281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:04 dist-test-slave-4qw6 krb5kdc[25281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
W20260619 02:30:04.313364 25375 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:04.305400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41358 (local address 127.17.96.254:37977)
0619 02:30:04.305560 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:04.305564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:04.305579 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:30:04.305629 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:04.305632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:04.305679 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:04.305839 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:04.306674 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.307266 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.308341 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.308764 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.308876 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.313162 (+ 4286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41358: BlockingRecv error: recv got EOF from 127.0.0.1:41358 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Jun 19 02:30:04 dist-test-slave-4qw6 krb5kdc[25281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836204, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM
I20260619 02:30:04.325699 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.315948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41374 (local address 127.17.96.254:37977)
0619 02:30:04.316158 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:04.316163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:04.316335 (+ 172us) server_negotiation.cc:407] Connection header received
0619 02:30:04.316517 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:04.316521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:04.316579 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:04.316674 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:04.317359 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.317989 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.318903 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.319149 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.319275 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.322154 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:04.322157 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:04.322160 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:30:04.322193 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:04.323723 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:04.324253 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:04.324259 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:04.324261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:04.324353 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:04.324678 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:04.324682 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:04.324684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:04.324913 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:04.325077 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:04.325356 (+ 279us) server_negotiation.cc:299] Negotiation successful
0619 02:30:04.325491 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 02:30:04.329057 25324 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:30:04.332590 25324 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:30:04.350520 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.343394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37476 (local address 127.17.96.195:42093)
0619 02:30:04.344061 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:04.344077 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:04.344108 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:30:04.344198 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:04.344206 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:04.344439 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:04.344616 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:30:04.345161 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.346471 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.348739 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.349526 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.349679 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.349980 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:04.350161 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 02:30:04.350295 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":161,"threads_started":1}
I20260619 02:30:04.350840 25833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.343984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:42291 (local address 127.0.0.1:33304)
0619 02:30:04.344543 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:04.344560 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:04.344668 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:04.345609 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:04.345613 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:04.345632 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:30:04.345790 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.345802 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.347676 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.347680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.348820 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.348832 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.350106 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.350113 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.350310 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.350384 (+ 74us) client_negotiation.cc:769] Sending connection context
0619 02:30:04.350466 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 02:30:04.350598 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"mutex_wait_us":157,"thread_start_us":107,"threads_started":1}
I20260619 02:30:04.351193 25835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.344402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33948 (local address 127.17.96.193:37233)
0619 02:30:04.344811 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:04.344819 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:04.345111 (+ 292us) server_negotiation.cc:407] Connection header received
0619 02:30:04.345211 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:04.345217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:04.345421 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:04.345602 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:30:04.346165 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.347258 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.348768 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.349715 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.349887 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.350096 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:04.350922 (+ 826us) server_negotiation.cc:299] Negotiation successful
0619 02:30:04.351059 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":82,"threads_started":1}
I20260619 02:30:04.350840 25834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.344147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:37233 (local address 127.0.0.1:33948)
0619 02:30:04.344880 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:04.344897 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:04.345002 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:04.345920 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:04.345927 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:04.345951 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:30:04.346043 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.346050 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.347411 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.347414 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.348660 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.348672 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.350151 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.350155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.350385 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.350403 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 02:30:04.350467 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 02:30:04.350598 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"mutex_wait_us":90,"thread_start_us":118,"threads_started":1}
I20260619 02:30:04.351639 25836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.344670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33304 (local address 127.17.96.194:42291)
0619 02:30:04.345092 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:04.345106 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:04.345129 (+ 23us) server_negotiation.cc:407] Connection header received
0619 02:30:04.345211 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:04.345217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:04.345421 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:04.345993 (+ 572us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:30:04.346158 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.347469 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.349001 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.349921 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.351069 (+ 1148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.351279 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:04.351342 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 02:30:04.351466 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":132,"threads_started":1}
I20260619 02:30:04.351955 25831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.343281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:42093 (local address 127.0.0.1:37476)
0619 02:30:04.343749 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:04.343808 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:04.343971 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:04.344816 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:04.344826 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:04.344854 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:30:04.345024 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.345033 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.346848 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.346851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.348537 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.348551 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.349686 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.349689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.349874 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.349920 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 02:30:04.350119 (+ 199us) client_negotiation.cc:240] Negotiation successful
0619 02:30:04.350326 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":41,"thread_start_us":144,"threads_started":1}
I20260619 02:30:04.356209 25743 tablet_service.cc:1467] Processing CreateTablet for tablet 396118b9672b43559151c70f87adc5d5 (DEFAULT_TABLE table=test-table [id=5fa6914f97804193933d44eb750035ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:30:04.356209 25603 tablet_service.cc:1467] Processing CreateTablet for tablet 396118b9672b43559151c70f87adc5d5 (DEFAULT_TABLE table=test-table [id=5fa6914f97804193933d44eb750035ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:30:04.356335 25463 tablet_service.cc:1467] Processing CreateTablet for tablet 396118b9672b43559151c70f87adc5d5 (DEFAULT_TABLE table=test-table [id=5fa6914f97804193933d44eb750035ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:30:04.357695 25743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 396118b9672b43559151c70f87adc5d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:04.357690 25603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 396118b9672b43559151c70f87adc5d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:04.357713 25463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 396118b9672b43559151c70f87adc5d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:04.368077 25839 tablet_bootstrap.cc:492] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a: Bootstrap starting.
I20260619 02:30:04.369035 25838 tablet_bootstrap.cc:492] T 396118b9672b43559151c70f87adc5d5 P 393b9d3c902f4989b620d5db1cc80e01: Bootstrap starting.
I20260619 02:30:04.370098 25837 tablet_bootstrap.cc:492] T 396118b9672b43559151c70f87adc5d5 P d1d857a119474703901ea085386d00c5: Bootstrap starting.
I20260619 02:30:04.370848 25839 tablet_bootstrap.cc:654] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:04.371192 25838 tablet_bootstrap.cc:654] T 396118b9672b43559151c70f87adc5d5 P 393b9d3c902f4989b620d5db1cc80e01: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:04.371841 25839 log.cc:824] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:30:04.371897 25838 log.cc:824] T 396118b9672b43559151c70f87adc5d5 P 393b9d3c902f4989b620d5db1cc80e01: Log is configured to *not* fsync() on all Append() calls
I20260619 02:30:04.372000 25837 tablet_bootstrap.cc:654] T 396118b9672b43559151c70f87adc5d5 P d1d857a119474703901ea085386d00c5: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:04.372871 25837 log.cc:824] T 396118b9672b43559151c70f87adc5d5 P d1d857a119474703901ea085386d00c5: Log is configured to *not* fsync() on all Append() calls
I20260619 02:30:04.374425 25839 tablet_bootstrap.cc:492] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a: No bootstrap required, opened a new log
I20260619 02:30:04.374425 25838 tablet_bootstrap.cc:492] T 396118b9672b43559151c70f87adc5d5 P 393b9d3c902f4989b620d5db1cc80e01: No bootstrap required, opened a new log
I20260619 02:30:04.374660 25839 ts_tablet_manager.cc:1397] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260619 02:30:04.374660 25838 ts_tablet_manager.cc:1397] T 396118b9672b43559151c70f87adc5d5 P 393b9d3c902f4989b620d5db1cc80e01: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 02:30:04.374794 25837 tablet_bootstrap.cc:492] T 396118b9672b43559151c70f87adc5d5 P d1d857a119474703901ea085386d00c5: No bootstrap required, opened a new log
I20260619 02:30:04.374995 25837 ts_tablet_manager.cc:1397] T 396118b9672b43559151c70f87adc5d5 P d1d857a119474703901ea085386d00c5: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 02:30:04.379104 25837 raft_consensus.cc:348] T 396118b9672b43559151c70f87adc5d5 P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } } peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } }
I20260619 02:30:04.379153 25839 raft_consensus.cc:348] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } } peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } }
I20260619 02:30:04.379153 25838 raft_consensus.cc:348] T 396118b9672b43559151c70f87adc5d5 P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } } peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } }
I20260619 02:30:04.379442 25837 raft_consensus.cc:374] T 396118b9672b43559151c70f87adc5d5 P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:04.379455 25838 raft_consensus.cc:374] T 396118b9672b43559151c70f87adc5d5 P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:04.379455 25839 raft_consensus.cc:374] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:04.379515 25837 raft_consensus.cc:729] T 396118b9672b43559151c70f87adc5d5 P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1d857a119474703901ea085386d00c5, State: Initialized, Role: FOLLOWER
I20260619 02:30:04.379513 25838 raft_consensus.cc:729] T 396118b9672b43559151c70f87adc5d5 P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 393b9d3c902f4989b620d5db1cc80e01, State: Initialized, Role: FOLLOWER
I20260619 02:30:04.379531 25839 raft_consensus.cc:729] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fce9948443c4324a28308ed2586472a, State: Initialized, Role: FOLLOWER
I20260619 02:30:04.380066 25837 consensus_queue.cc:260] T 396118b9672b43559151c70f87adc5d5 P d1d857a119474703901ea085386d00c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } } peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } }
I20260619 02:30:04.380072 25839 consensus_queue.cc:260] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } } peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } }
I20260619 02:30:04.380072 25838 consensus_queue.cc:260] T 396118b9672b43559151c70f87adc5d5 P 393b9d3c902f4989b620d5db1cc80e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } } peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } }
I20260619 02:30:04.380906 25529 heartbeater.cc:503] Master 127.17.96.254:37977 was elected leader, sending a full tablet report...
I20260619 02:30:04.381374 25837 ts_tablet_manager.cc:1428] T 396118b9672b43559151c70f87adc5d5 P d1d857a119474703901ea085386d00c5: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260619 02:30:04.381376 25839 ts_tablet_manager.cc:1428] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260619 02:30:04.381376 25838 ts_tablet_manager.cc:1428] T 396118b9672b43559151c70f87adc5d5 P 393b9d3c902f4989b620d5db1cc80e01: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260619 02:30:04.381809 25669 heartbeater.cc:503] Master 127.17.96.254:37977 was elected leader, sending a full tablet report...
I20260619 02:30:04.381946 25809 heartbeater.cc:503] Master 127.17.96.254:37977 was elected leader, sending a full tablet report...
I20260619 02:30:04.480877 25844 raft_consensus.cc:482] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:30:04.481138 25844 raft_consensus.cc:504] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } } peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } }
I20260619 02:30:04.482555 25844 leader_election.cc:288] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1d857a119474703901ea085386d00c5 (127.17.96.193:37233), 393b9d3c902f4989b620d5db1cc80e01 (127.17.96.194:42291)
I20260619 02:30:04.486991 25702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.482967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:37233 (local address 127.17.96.195:52801)
0619 02:30:04.483099 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:04.483118 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:04.483232 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:04.483524 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:04.483527 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:04.483546 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:30:04.483689 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.483697 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.484660 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.484666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.485799 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.485810 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.486539 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.486543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.486689 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.486713 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 02:30:04.486777 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 02:30:04.486851 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":6}
I20260619 02:30:04.487066 25835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.483065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:52801 (local address 127.17.96.193:37233)
0619 02:30:04.483235 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:04.483239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:04.483254 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:30:04.483313 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:04.483315 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:04.483360 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:04.483487 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:30:04.483821 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.484517 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.485943 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.486402 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.486512 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.486569 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:04.486825 (+ 256us) server_negotiation.cc:299] Negotiation successful
0619 02:30:04.486921 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 02:30:04.487923 25846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.482969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:42291 (local address 127.17.96.195:38373)
0619 02:30:04.483789 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:04.483805 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:04.483909 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:04.484174 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:04.484176 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:04.484189 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:30:04.484716 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.484724 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.485518 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.485522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.486537 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.486546 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.487495 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.487500 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.487611 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.487647 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 02:30:04.487729 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 02:30:04.487785 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"mutex_wait_us":351,"thread_start_us":168,"threads_started":1}
I20260619 02:30:04.487998 25836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.483017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:38373 (local address 127.17.96.194:42291)
0619 02:30:04.483166 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:04.483171 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:04.483901 (+ 730us) server_negotiation.cc:407] Connection header received
0619 02:30:04.483986 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:04.483990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:04.484065 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:04.484153 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:30:04.484852 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.485382 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.486673 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.487320 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.487466 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.487518 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:04.487779 (+ 261us) server_negotiation.cc:299] Negotiation successful
0619 02:30:04.487870 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:30:04.488023 25483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "396118b9672b43559151c70f87adc5d5" candidate_uuid: "6fce9948443c4324a28308ed2586472a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1d857a119474703901ea085386d00c5" is_pre_election: true
I20260619 02:30:04.488443 25483 raft_consensus.cc:2454] T 396118b9672b43559151c70f87adc5d5 P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fce9948443c4324a28308ed2586472a in term 0.
I20260619 02:30:04.488664 25623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "396118b9672b43559151c70f87adc5d5" candidate_uuid: "6fce9948443c4324a28308ed2586472a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "393b9d3c902f4989b620d5db1cc80e01" is_pre_election: true
I20260619 02:30:04.488971 25623 raft_consensus.cc:2454] T 396118b9672b43559151c70f87adc5d5 P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fce9948443c4324a28308ed2586472a in term 0.
I20260619 02:30:04.489113 25694 leader_election.cc:302] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fce9948443c4324a28308ed2586472a, d1d857a119474703901ea085386d00c5; no voters:
I20260619 02:30:04.489440 25844 raft_consensus.cc:2781] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:30:04.489534 25844 raft_consensus.cc:482] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:30:04.489583 25844 raft_consensus.cc:3037] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:04.490816 25844 raft_consensus.cc:504] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } } peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } }
I20260619 02:30:04.491232 25844 leader_election.cc:288] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a [CANDIDATE]: Term 1 election: Requested vote from peers d1d857a119474703901ea085386d00c5 (127.17.96.193:37233), 393b9d3c902f4989b620d5db1cc80e01 (127.17.96.194:42291)
I20260619 02:30:04.491673 25483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "396118b9672b43559151c70f87adc5d5" candidate_uuid: "6fce9948443c4324a28308ed2586472a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1d857a119474703901ea085386d00c5"
I20260619 02:30:04.491699 25623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "396118b9672b43559151c70f87adc5d5" candidate_uuid: "6fce9948443c4324a28308ed2586472a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "393b9d3c902f4989b620d5db1cc80e01"
I20260619 02:30:04.491798 25483 raft_consensus.cc:3037] T 396118b9672b43559151c70f87adc5d5 P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:04.491853 25623 raft_consensus.cc:3037] T 396118b9672b43559151c70f87adc5d5 P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:04.492938 25483 raft_consensus.cc:2454] T 396118b9672b43559151c70f87adc5d5 P d1d857a119474703901ea085386d00c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fce9948443c4324a28308ed2586472a in term 1.
I20260619 02:30:04.493089 25623 raft_consensus.cc:2454] T 396118b9672b43559151c70f87adc5d5 P 393b9d3c902f4989b620d5db1cc80e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fce9948443c4324a28308ed2586472a in term 1.
I20260619 02:30:04.493328 25694 leader_election.cc:302] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fce9948443c4324a28308ed2586472a, d1d857a119474703901ea085386d00c5; no voters:
I20260619 02:30:04.493534 25844 raft_consensus.cc:2781] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:30:04.493849 25844 raft_consensus.cc:686] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a [term 1 LEADER]: Becoming Leader. State: Replica: 6fce9948443c4324a28308ed2586472a, State: Running, Role: LEADER
I20260619 02:30:04.494246 25844 consensus_queue.cc:237] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } } peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } }
I20260619 02:30:04.498250 25324 catalog_manager.cc:5495] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fce9948443c4324a28308ed2586472a (127.17.96.195). New cstate: current_term: 1 leader_uuid: "6fce9948443c4324a28308ed2586472a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } health_report { overall_health: UNKNOWN } } }
W20260619 02:30:04.499336 25670 tablet.cc:2369] T 396118b9672b43559151c70f87adc5d5 P 393b9d3c902f4989b620d5db1cc80e01: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:30:04.499891 25530 tablet.cc:2369] T 396118b9672b43559151c70f87adc5d5 P d1d857a119474703901ea085386d00c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:30:04.522874 25810 tablet.cc:2369] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:30:04.549031 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.544805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37478 (local address 127.17.96.195:42093)
0619 02:30:04.544970 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:04.544977 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:04.544998 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:30:04.545107 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:04.545110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:04.545171 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:04.545310 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:30:04.545592 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.546204 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.547252 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.547513 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.547645 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.547961 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:30:04.548494 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:30:04.548686 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:04.548782 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 02:30:04.548853 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 02:30:04.556579 25623 raft_consensus.cc:1261] T 396118b9672b43559151c70f87adc5d5 P 393b9d3c902f4989b620d5db1cc80e01 [term 1 FOLLOWER]: Refusing update from remote peer 6fce9948443c4324a28308ed2586472a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:30:04.556579 25483 raft_consensus.cc:1261] T 396118b9672b43559151c70f87adc5d5 P d1d857a119474703901ea085386d00c5 [term 1 FOLLOWER]: Refusing update from remote peer 6fce9948443c4324a28308ed2586472a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:30:04.557456 25847 consensus_queue.cc:1035] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a [LEADER]: Connected to new peer: Peer: permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:30:04.557685 25844 consensus_queue.cc:1035] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:30:04.569347 25852 mvcc.cc:205] Tried to move back new op lower bound from 7298401093854912512 to 7298401093611520000. Current Snapshot: MvccSnapshot[applied={T|T < 7298401093854912512}]
I20260619 02:30:04.581048 25326 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:30:04.581287 25326 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:30:04.584616 25326 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272: Sending DeleteTablet for 3 replicas of tablet 396118b9672b43559151c70f87adc5d5
I20260619 02:30:04.585850 25743 tablet_service.cc:1514] Processing DeleteTablet for tablet 396118b9672b43559151c70f87adc5d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:30:04 UTC) from {username='oryx', principal='oryx/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:37476
I20260619 02:30:04.585924 25603 tablet_service.cc:1514] Processing DeleteTablet for tablet 396118b9672b43559151c70f87adc5d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:30:04 UTC) from {username='oryx', principal='oryx/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:33304
I20260619 02:30:04.586021 25463 tablet_service.cc:1514] Processing DeleteTablet for tablet 396118b9672b43559151c70f87adc5d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:30:04 UTC) from {username='oryx', principal='oryx/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:33948
I20260619 02:30:04.586503 25860 tablet_replica.cc:330] T 396118b9672b43559151c70f87adc5d5 P 393b9d3c902f4989b620d5db1cc80e01: stopping tablet replica
I20260619 02:30:04.586726 25862 tablet_replica.cc:330] T 396118b9672b43559151c70f87adc5d5 P d1d857a119474703901ea085386d00c5: stopping tablet replica
I20260619 02:30:04.586941 25860 raft_consensus.cc:2229] T 396118b9672b43559151c70f87adc5d5 P 393b9d3c902f4989b620d5db1cc80e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:30:04.587193 25862 raft_consensus.cc:2229] T 396118b9672b43559151c70f87adc5d5 P d1d857a119474703901ea085386d00c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:30:04.587380 25860 raft_consensus.cc:2258] T 396118b9672b43559151c70f87adc5d5 P 393b9d3c902f4989b620d5db1cc80e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:30:04.587587 25326 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 02:30:04.587894 25861 tablet_replica.cc:330] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a: stopping tablet replica
I20260619 02:30:04.587618 25862 raft_consensus.cc:2258] T 396118b9672b43559151c70f87adc5d5 P d1d857a119474703901ea085386d00c5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 02:30:04.588155 25326 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:30:04.588289 25861 raft_consensus.cc:2229] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:30:04.588832 25861 raft_consensus.cc:2258] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:30:04.590325 25862 ts_tablet_manager.cc:1905] T 396118b9672b43559151c70f87adc5d5 P d1d857a119474703901ea085386d00c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:30:04.590420 25861 ts_tablet_manager.cc:1905] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:30:04.593849 25862 ts_tablet_manager.cc:1918] T 396118b9672b43559151c70f87adc5d5 P d1d857a119474703901ea085386d00c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:30:04.593849 25861 ts_tablet_manager.cc:1918] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:30:04.594254 25862 log.cc:1196] T 396118b9672b43559151c70f87adc5d5 P d1d857a119474703901ea085386d00c5: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-0/wal/wals/396118b9672b43559151c70f87adc5d5
I20260619 02:30:04.594864 25862 ts_tablet_manager.cc:1939] T 396118b9672b43559151c70f87adc5d5 P d1d857a119474703901ea085386d00c5: Deleting consensus metadata
I20260619 02:30:04.594964 25860 ts_tablet_manager.cc:1905] T 396118b9672b43559151c70f87adc5d5 P 393b9d3c902f4989b620d5db1cc80e01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:30:04.595490 25743 tablet_service.cc:1467] Processing CreateTablet for tablet d862bc488d4e4eeba4b40d939afdcd65 (DEFAULT_TABLE table=test-table [id=54607fd398b343348496da651e5de976]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:30:04.595917 25743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d862bc488d4e4eeba4b40d939afdcd65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:04.598240 25310 catalog_manager.cc:4841] TS d1d857a119474703901ea085386d00c5 (127.17.96.193:37233): tablet 396118b9672b43559151c70f87adc5d5 (table test-table [id=5fa6914f97804193933d44eb750035ab]) successfully deleted
I20260619 02:30:04.598371 25463 tablet_service.cc:1467] Processing CreateTablet for tablet d862bc488d4e4eeba4b40d939afdcd65 (DEFAULT_TABLE table=test-table [id=54607fd398b343348496da651e5de976]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:30:04.598722 25463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d862bc488d4e4eeba4b40d939afdcd65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:04.599470 25860 ts_tablet_manager.cc:1918] T 396118b9672b43559151c70f87adc5d5 P 393b9d3c902f4989b620d5db1cc80e01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:30:04.593997 25861 log.cc:1196] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/396118b9672b43559151c70f87adc5d5
I20260619 02:30:04.599609 25860 log.cc:1196] T 396118b9672b43559151c70f87adc5d5 P 393b9d3c902f4989b620d5db1cc80e01: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/396118b9672b43559151c70f87adc5d5
I20260619 02:30:04.600369 25861 ts_tablet_manager.cc:1939] T 396118b9672b43559151c70f87adc5d5 P 6fce9948443c4324a28308ed2586472a: Deleting consensus metadata
I20260619 02:30:04.599970 25860 ts_tablet_manager.cc:1939] T 396118b9672b43559151c70f87adc5d5 P 393b9d3c902f4989b620d5db1cc80e01: Deleting consensus metadata
I20260619 02:30:04.602799 25311 catalog_manager.cc:4841] TS 393b9d3c902f4989b620d5db1cc80e01 (127.17.96.194:42291): tablet 396118b9672b43559151c70f87adc5d5 (table test-table [id=5fa6914f97804193933d44eb750035ab]) successfully deleted
I20260619 02:30:04.603039 25310 catalog_manager.cc:4841] TS 6fce9948443c4324a28308ed2586472a (127.17.96.195:42093): tablet 396118b9672b43559151c70f87adc5d5 (table test-table [id=5fa6914f97804193933d44eb750035ab]) successfully deleted
I20260619 02:30:04.603864 25603 tablet_service.cc:1467] Processing CreateTablet for tablet d862bc488d4e4eeba4b40d939afdcd65 (DEFAULT_TABLE table=test-table [id=54607fd398b343348496da651e5de976]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:30:04.604321 25603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d862bc488d4e4eeba4b40d939afdcd65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:04.605060 25839 tablet_bootstrap.cc:492] T d862bc488d4e4eeba4b40d939afdcd65 P 6fce9948443c4324a28308ed2586472a: Bootstrap starting.
I20260619 02:30:04.605252 25837 tablet_bootstrap.cc:492] T d862bc488d4e4eeba4b40d939afdcd65 P d1d857a119474703901ea085386d00c5: Bootstrap starting.
I20260619 02:30:04.606472 25839 tablet_bootstrap.cc:654] T d862bc488d4e4eeba4b40d939afdcd65 P 6fce9948443c4324a28308ed2586472a: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:04.606634 25837 tablet_bootstrap.cc:654] T d862bc488d4e4eeba4b40d939afdcd65 P d1d857a119474703901ea085386d00c5: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:04.608086 25838 tablet_bootstrap.cc:492] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01: Bootstrap starting.
I20260619 02:30:04.608453 25837 tablet_bootstrap.cc:492] T d862bc488d4e4eeba4b40d939afdcd65 P d1d857a119474703901ea085386d00c5: No bootstrap required, opened a new log
I20260619 02:30:04.608623 25837 ts_tablet_manager.cc:1397] T d862bc488d4e4eeba4b40d939afdcd65 P d1d857a119474703901ea085386d00c5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 02:30:04.608952 25839 tablet_bootstrap.cc:492] T d862bc488d4e4eeba4b40d939afdcd65 P 6fce9948443c4324a28308ed2586472a: No bootstrap required, opened a new log
I20260619 02:30:04.609061 25839 ts_tablet_manager.cc:1397] T d862bc488d4e4eeba4b40d939afdcd65 P 6fce9948443c4324a28308ed2586472a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 02:30:04.609346 25838 tablet_bootstrap.cc:654] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:04.609637 25839 raft_consensus.cc:348] T d862bc488d4e4eeba4b40d939afdcd65 P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:04.609637 25837 raft_consensus.cc:348] T d862bc488d4e4eeba4b40d939afdcd65 P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:04.609828 25839 raft_consensus.cc:374] T d862bc488d4e4eeba4b40d939afdcd65 P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:04.609844 25837 raft_consensus.cc:374] T d862bc488d4e4eeba4b40d939afdcd65 P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:04.609895 25837 raft_consensus.cc:729] T d862bc488d4e4eeba4b40d939afdcd65 P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1d857a119474703901ea085386d00c5, State: Initialized, Role: FOLLOWER
I20260619 02:30:04.609895 25839 raft_consensus.cc:729] T d862bc488d4e4eeba4b40d939afdcd65 P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fce9948443c4324a28308ed2586472a, State: Initialized, Role: FOLLOWER
I20260619 02:30:04.610066 25839 consensus_queue.cc:260] T d862bc488d4e4eeba4b40d939afdcd65 P 6fce9948443c4324a28308ed2586472a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:04.610109 25837 consensus_queue.cc:260] T d862bc488d4e4eeba4b40d939afdcd65 P d1d857a119474703901ea085386d00c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:04.610407 25839 ts_tablet_manager.cc:1428] T d862bc488d4e4eeba4b40d939afdcd65 P 6fce9948443c4324a28308ed2586472a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260619 02:30:04.612193 25837 ts_tablet_manager.cc:1428] T d862bc488d4e4eeba4b40d939afdcd65 P d1d857a119474703901ea085386d00c5: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260619 02:30:04.613554 25838 tablet_bootstrap.cc:492] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01: No bootstrap required, opened a new log
I20260619 02:30:04.613713 25838 ts_tablet_manager.cc:1397] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260619 02:30:04.614449 25838 raft_consensus.cc:348] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:04.614650 25838 raft_consensus.cc:374] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:04.614692 25838 raft_consensus.cc:729] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 393b9d3c902f4989b620d5db1cc80e01, State: Initialized, Role: FOLLOWER
I20260619 02:30:04.614859 25838 consensus_queue.cc:260] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:04.615228 25838 ts_tablet_manager.cc:1428] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:30:04.671634 25845 raft_consensus.cc:482] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:30:04.671871 25845 raft_consensus.cc:504] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:04.673398 25845 leader_election.cc:288] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1d857a119474703901ea085386d00c5 (127.17.96.193:37233), 6fce9948443c4324a28308ed2586472a (127.17.96.195:42093)
I20260619 02:30:04.679340 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.674395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:55953 (local address 127.17.96.195:42093)
0619 02:30:04.674532 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:04.674537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:04.674693 (+ 156us) server_negotiation.cc:407] Connection header received
0619 02:30:04.674806 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:04.674810 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:04.674872 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:04.674989 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:30:04.675247 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.675805 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.677779 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.678444 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.678549 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.678606 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:04.679084 (+ 478us) server_negotiation.cc:299] Negotiation successful
0619 02:30:04.679179 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:30:04.679392 25867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.674232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:42093 (local address 127.17.96.194:55953)
0619 02:30:04.674586 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:04.674601 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:04.674700 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:04.675011 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:04.675016 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:04.675032 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:30:04.675142 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.675151 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.675945 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.675949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.677534 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.677548 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.678763 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.678767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.678909 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.678964 (+ 55us) client_negotiation.cc:769] Sending connection context
0619 02:30:04.679042 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 02:30:04.679174 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":168,"thread_start_us":113,"threads_started":1}
I20260619 02:30:04.679404 25866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.673817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:37233 (local address 127.17.96.194:49617)
0619 02:30:04.674194 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:04.674212 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:04.674420 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:04.674875 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:04.674877 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:04.674892 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:30:04.675065 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.675072 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.676151 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.676154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.677534 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.677548 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.678736 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.678743 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.678896 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.678958 (+ 62us) client_negotiation.cc:769] Sending connection context
0619 02:30:04.679042 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 02:30:04.679174 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":105,"thread_start_us":94,"threads_started":1}
I20260619 02:30:04.679920 25835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.673975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:49617 (local address 127.17.96.193:37233)
0619 02:30:04.674155 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:04.674161 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:04.674302 (+ 141us) server_negotiation.cc:407] Connection header received
0619 02:30:04.674575 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:04.674579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:04.674647 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:04.674704 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:30:04.675199 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.676019 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.677781 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.678444 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.678549 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.678606 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:04.679639 (+ 1033us) server_negotiation.cc:299] Negotiation successful
0619 02:30:04.679742 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 02:30:04.680425 25483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d862bc488d4e4eeba4b40d939afdcd65" candidate_uuid: "393b9d3c902f4989b620d5db1cc80e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1d857a119474703901ea085386d00c5" is_pre_election: true
I20260619 02:30:04.680636 25483 raft_consensus.cc:2454] T d862bc488d4e4eeba4b40d939afdcd65 P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 393b9d3c902f4989b620d5db1cc80e01 in term 0.
I20260619 02:30:04.680763 25763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d862bc488d4e4eeba4b40d939afdcd65" candidate_uuid: "393b9d3c902f4989b620d5db1cc80e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fce9948443c4324a28308ed2586472a" is_pre_election: true
I20260619 02:30:04.681039 25763 raft_consensus.cc:2454] T d862bc488d4e4eeba4b40d939afdcd65 P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 393b9d3c902f4989b620d5db1cc80e01 in term 0.
I20260619 02:30:04.681061 25554 leader_election.cc:302] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 393b9d3c902f4989b620d5db1cc80e01, d1d857a119474703901ea085386d00c5; no voters:
I20260619 02:30:04.681347 25845 raft_consensus.cc:2781] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:30:04.681416 25845 raft_consensus.cc:482] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:30:04.681466 25845 raft_consensus.cc:3037] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:04.682760 25845 raft_consensus.cc:504] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:04.683250 25845 leader_election.cc:288] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01 [CANDIDATE]: Term 1 election: Requested vote from peers d1d857a119474703901ea085386d00c5 (127.17.96.193:37233), 6fce9948443c4324a28308ed2586472a (127.17.96.195:42093)
I20260619 02:30:04.683779 25763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d862bc488d4e4eeba4b40d939afdcd65" candidate_uuid: "393b9d3c902f4989b620d5db1cc80e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fce9948443c4324a28308ed2586472a"
I20260619 02:30:04.683804 25483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d862bc488d4e4eeba4b40d939afdcd65" candidate_uuid: "393b9d3c902f4989b620d5db1cc80e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1d857a119474703901ea085386d00c5"
I20260619 02:30:04.683951 25483 raft_consensus.cc:3037] T d862bc488d4e4eeba4b40d939afdcd65 P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:04.683947 25763 raft_consensus.cc:3037] T d862bc488d4e4eeba4b40d939afdcd65 P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:04.685140 25483 raft_consensus.cc:2454] T d862bc488d4e4eeba4b40d939afdcd65 P d1d857a119474703901ea085386d00c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 393b9d3c902f4989b620d5db1cc80e01 in term 1.
I20260619 02:30:04.685534 25554 leader_election.cc:302] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 393b9d3c902f4989b620d5db1cc80e01, d1d857a119474703901ea085386d00c5; no voters:
I20260619 02:30:04.685590 25763 raft_consensus.cc:2454] T d862bc488d4e4eeba4b40d939afdcd65 P 6fce9948443c4324a28308ed2586472a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 393b9d3c902f4989b620d5db1cc80e01 in term 1.
I20260619 02:30:04.685779 25845 raft_consensus.cc:2781] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:30:04.686084 25845 raft_consensus.cc:686] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01 [term 1 LEADER]: Becoming Leader. State: Replica: 393b9d3c902f4989b620d5db1cc80e01, State: Running, Role: LEADER
I20260619 02:30:04.686427 25845 consensus_queue.cc:237] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:04.689664 25326 catalog_manager.cc:5495] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 393b9d3c902f4989b620d5db1cc80e01 (127.17.96.194). New cstate: current_term: 1 leader_uuid: "393b9d3c902f4989b620d5db1cc80e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } health_report { overall_health: UNKNOWN } } }
Jun 19 02:30:04 dist-test-slave-4qw6 krb5kdc[25281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836203, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.254@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM
I20260619 02:30:04.732621 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.720604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41380 (local address 127.17.96.254:37977)
0619 02:30:04.720759 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:04.720764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:04.721195 (+ 431us) server_negotiation.cc:407] Connection header received
0619 02:30:04.721462 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:04.721465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:04.721530 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:04.721642 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:04.722952 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.723617 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.725140 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.725398 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.725503 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.728872 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:04.728878 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:04.728887 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:30:04.728910 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:04.730583 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:04.731155 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:04.731161 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:04.731163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:04.731243 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:04.731555 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:04.731559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:04.731560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:04.731768 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:04.731892 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:04.732250 (+ 358us) server_negotiation.cc:299] Negotiation successful
0619 02:30:04.732420 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 02:30:04.732647 25875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.720369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:37977 (local address 127.0.0.1:41380)
0619 02:30:04.720929 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:04.720953 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:04.721301 (+ 348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:04.721715 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:04.721718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:04.722525 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:04.722716 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.722731 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.723807 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.723811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.724947 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.724968 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.725577 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.725581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.725682 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.726719 (+ 1037us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:04.726738 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:04.728681 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:30:04.730759 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:04.730771 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:04.730778 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:04.731044 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:04.731359 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:04.731362 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:04.731364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:04.731449 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:04.731891 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:30:04.731900 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:30:04.732066 (+ 166us) client_negotiation.cc:769] Sending connection context
0619 02:30:04.732193 (+ 127us) client_negotiation.cc:240] Negotiation successful
0619 02:30:04.732497 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":106,"thread_start_us":207,"threads_started":1}
I20260619 02:30:04.738004 25326 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:41380:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 02:30:04.739063 25326 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:30:04.746747 25463 tablet_service.cc:1467] Processing CreateTablet for tablet 3709c4bf47fe447198a270c8a07def72 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=92f287a339a14f7bbb49c1e29306bbaa]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:30:04.747346 25463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3709c4bf47fe447198a270c8a07def72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:04.747706 25603 tablet_service.cc:1467] Processing CreateTablet for tablet 3709c4bf47fe447198a270c8a07def72 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=92f287a339a14f7bbb49c1e29306bbaa]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:30:04.748135 25603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3709c4bf47fe447198a270c8a07def72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:04.751140 25743 tablet_service.cc:1467] Processing CreateTablet for tablet 3709c4bf47fe447198a270c8a07def72 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=92f287a339a14f7bbb49c1e29306bbaa]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:30:04.751348 25837 tablet_bootstrap.cc:492] T 3709c4bf47fe447198a270c8a07def72 P d1d857a119474703901ea085386d00c5: Bootstrap starting.
I20260619 02:30:04.751596 25743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3709c4bf47fe447198a270c8a07def72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:04.752007 25838 tablet_bootstrap.cc:492] T 3709c4bf47fe447198a270c8a07def72 P 393b9d3c902f4989b620d5db1cc80e01: Bootstrap starting.
I20260619 02:30:04.752653 25837 tablet_bootstrap.cc:654] T 3709c4bf47fe447198a270c8a07def72 P d1d857a119474703901ea085386d00c5: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:04.753232 25838 tablet_bootstrap.cc:654] T 3709c4bf47fe447198a270c8a07def72 P 393b9d3c902f4989b620d5db1cc80e01: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:04.754840 25838 tablet_bootstrap.cc:492] T 3709c4bf47fe447198a270c8a07def72 P 393b9d3c902f4989b620d5db1cc80e01: No bootstrap required, opened a new log
I20260619 02:30:04.754972 25838 ts_tablet_manager.cc:1397] T 3709c4bf47fe447198a270c8a07def72 P 393b9d3c902f4989b620d5db1cc80e01: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.002s
I20260619 02:30:04.755522 25838 raft_consensus.cc:348] T 3709c4bf47fe447198a270c8a07def72 P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:04.755697 25838 raft_consensus.cc:374] T 3709c4bf47fe447198a270c8a07def72 P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:04.755750 25838 raft_consensus.cc:729] T 3709c4bf47fe447198a270c8a07def72 P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 393b9d3c902f4989b620d5db1cc80e01, State: Initialized, Role: FOLLOWER
I20260619 02:30:04.755939 25838 consensus_queue.cc:260] T 3709c4bf47fe447198a270c8a07def72 P 393b9d3c902f4989b620d5db1cc80e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:04.756309 25868 tablet_replica.cc:439] T 3709c4bf47fe447198a270c8a07def72 P 393b9d3c902f4989b620d5db1cc80e01: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } } }
I20260619 02:30:04.756487 25868 tablet_replica.cc:442] T 3709c4bf47fe447198a270c8a07def72 P 393b9d3c902f4989b620d5db1cc80e01: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:30:04.756779 25838 ts_tablet_manager.cc:1428] T 3709c4bf47fe447198a270c8a07def72 P 393b9d3c902f4989b620d5db1cc80e01: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:30:04.757694 25839 tablet_bootstrap.cc:492] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a: Bootstrap starting.
I20260619 02:30:04.759178 25839 tablet_bootstrap.cc:654] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:04.770872 25837 tablet_bootstrap.cc:492] T 3709c4bf47fe447198a270c8a07def72 P d1d857a119474703901ea085386d00c5: No bootstrap required, opened a new log
I20260619 02:30:04.770891 25839 tablet_bootstrap.cc:492] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a: No bootstrap required, opened a new log
I20260619 02:30:04.771050 25839 ts_tablet_manager.cc:1397] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.003s
I20260619 02:30:04.771054 25837 ts_tablet_manager.cc:1397] T 3709c4bf47fe447198a270c8a07def72 P d1d857a119474703901ea085386d00c5: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20260619 02:30:04.771603 25837 raft_consensus.cc:348] T 3709c4bf47fe447198a270c8a07def72 P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:04.771628 25839 raft_consensus.cc:348] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:04.771763 25839 raft_consensus.cc:374] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:04.771766 25837 raft_consensus.cc:374] T 3709c4bf47fe447198a270c8a07def72 P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:04.771812 25839 raft_consensus.cc:729] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fce9948443c4324a28308ed2586472a, State: Initialized, Role: FOLLOWER
I20260619 02:30:04.771818 25837 raft_consensus.cc:729] T 3709c4bf47fe447198a270c8a07def72 P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1d857a119474703901ea085386d00c5, State: Initialized, Role: FOLLOWER
I20260619 02:30:04.771960 25839 consensus_queue.cc:260] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:04.771979 25837 consensus_queue.cc:260] T 3709c4bf47fe447198a270c8a07def72 P d1d857a119474703901ea085386d00c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:04.772276 25839 ts_tablet_manager.cc:1428] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260619 02:30:04.772291 25837 ts_tablet_manager.cc:1428] T 3709c4bf47fe447198a270c8a07def72 P d1d857a119474703901ea085386d00c5: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260619 02:30:04.772595 25843 tablet_replica.cc:439] T 3709c4bf47fe447198a270c8a07def72 P d1d857a119474703901ea085386d00c5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } } }
I20260619 02:30:04.772591 25855 tablet_replica.cc:439] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } } }
I20260619 02:30:04.772773 25855 tablet_replica.cc:442] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:30:04.772773 25843 tablet_replica.cc:442] T 3709c4bf47fe447198a270c8a07def72 P d1d857a119474703901ea085386d00c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:30:04.784080 25855 raft_consensus.cc:482] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:30:04.784301 25855 raft_consensus.cc:504] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:04.784999 25855 leader_election.cc:288] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1d857a119474703901ea085386d00c5 (127.17.96.193:37233), 393b9d3c902f4989b620d5db1cc80e01 (127.17.96.194:42291)
I20260619 02:30:04.785626 25483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3709c4bf47fe447198a270c8a07def72" candidate_uuid: "6fce9948443c4324a28308ed2586472a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1d857a119474703901ea085386d00c5" is_pre_election: true
I20260619 02:30:04.785864 25623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3709c4bf47fe447198a270c8a07def72" candidate_uuid: "6fce9948443c4324a28308ed2586472a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "393b9d3c902f4989b620d5db1cc80e01" is_pre_election: true
I20260619 02:30:04.785943 25483 raft_consensus.cc:2454] T 3709c4bf47fe447198a270c8a07def72 P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fce9948443c4324a28308ed2586472a in term 0.
I20260619 02:30:04.786067 25623 raft_consensus.cc:2454] T 3709c4bf47fe447198a270c8a07def72 P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fce9948443c4324a28308ed2586472a in term 0.
I20260619 02:30:04.786346 25694 leader_election.cc:302] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fce9948443c4324a28308ed2586472a, d1d857a119474703901ea085386d00c5; no voters:
I20260619 02:30:04.786607 25855 raft_consensus.cc:2781] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:30:04.786690 25855 raft_consensus.cc:482] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:30:04.786726 25855 raft_consensus.cc:3037] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:04.788185 25855 raft_consensus.cc:504] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:04.788726 25855 leader_election.cc:288] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a [CANDIDATE]: Term 1 election: Requested vote from peers d1d857a119474703901ea085386d00c5 (127.17.96.193:37233), 393b9d3c902f4989b620d5db1cc80e01 (127.17.96.194:42291)
I20260619 02:30:04.789150 25483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3709c4bf47fe447198a270c8a07def72" candidate_uuid: "6fce9948443c4324a28308ed2586472a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1d857a119474703901ea085386d00c5"
I20260619 02:30:04.789330 25483 raft_consensus.cc:3037] T 3709c4bf47fe447198a270c8a07def72 P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:04.789333 25623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3709c4bf47fe447198a270c8a07def72" candidate_uuid: "6fce9948443c4324a28308ed2586472a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "393b9d3c902f4989b620d5db1cc80e01"
I20260619 02:30:04.789515 25623 raft_consensus.cc:3037] T 3709c4bf47fe447198a270c8a07def72 P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:04.790998 25623 raft_consensus.cc:2454] T 3709c4bf47fe447198a270c8a07def72 P 393b9d3c902f4989b620d5db1cc80e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fce9948443c4324a28308ed2586472a in term 1.
I20260619 02:30:04.791000 25483 raft_consensus.cc:2454] T 3709c4bf47fe447198a270c8a07def72 P d1d857a119474703901ea085386d00c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fce9948443c4324a28308ed2586472a in term 1.
I20260619 02:30:04.791527 25694 leader_election.cc:302] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fce9948443c4324a28308ed2586472a, d1d857a119474703901ea085386d00c5; no voters:
I20260619 02:30:04.791868 25855 raft_consensus.cc:2781] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:30:04.791971 25855 raft_consensus.cc:686] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a [term 1 LEADER]: Becoming Leader. State: Replica: 6fce9948443c4324a28308ed2586472a, State: Running, Role: LEADER
I20260619 02:30:04.792448 25855 consensus_queue.cc:237] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:04.794127 25847 tablet_replica.cc:439] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a: TxnStatusTablet state changed. Reason: New leader 6fce9948443c4324a28308ed2586472a. Latest consensus state: current_term: 1 leader_uuid: "6fce9948443c4324a28308ed2586472a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } } }
I20260619 02:30:04.794330 25847 tablet_replica.cc:442] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:30:04.794694 25880 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 02:30:04.794903 25880 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 02:30:04.795709 25326 catalog_manager.cc:5495] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fce9948443c4324a28308ed2586472a (127.17.96.195). New cstate: current_term: 1 leader_uuid: "6fce9948443c4324a28308ed2586472a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } health_report { overall_health: HEALTHY } } }
I20260619 02:30:04.813557 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.808928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37486 (local address 127.17.96.195:42093)
0619 02:30:04.809110 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:04.809116 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:04.809138 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:30:04.809270 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:04.809274 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:04.809352 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:04.809459 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:30:04.809711 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.810581 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.811799 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.812199 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.812373 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.812911 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:30:04.813030 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:30:04.813133 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:04.813312 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 02:30:04.813386 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260619 02:30:04.813549 25875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.808826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:42093 (local address 127.0.0.1:37486)
0619 02:30:04.809039 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:04.809055 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:04.809157 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:04.809481 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:04.809483 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:04.809490 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:30:04.809593 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.809598 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.810843 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.810849 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.811594 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.811610 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.812422 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.812425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.812542 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.812606 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:30:04.813164 (+ 558us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:30:04.813184 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 02:30:04.813261 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 02:30:04.813366 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260619 02:30:04.817358 25623 raft_consensus.cc:1261] T 3709c4bf47fe447198a270c8a07def72 P 393b9d3c902f4989b620d5db1cc80e01 [term 1 FOLLOWER]: Refusing update from remote peer 6fce9948443c4324a28308ed2586472a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:30:04.817477 25483 raft_consensus.cc:1261] T 3709c4bf47fe447198a270c8a07def72 P d1d857a119474703901ea085386d00c5 [term 1 FOLLOWER]: Refusing update from remote peer 6fce9948443c4324a28308ed2586472a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:30:04.818015 25855 consensus_queue.cc:1035] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a [LEADER]: Connected to new peer: Peer: permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:30:04.818239 25847 consensus_queue.cc:1035] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:30:04.820443 25868 tablet_replica.cc:439] T 3709c4bf47fe447198a270c8a07def72 P 393b9d3c902f4989b620d5db1cc80e01: TxnStatusTablet state changed. Reason: New leader 6fce9948443c4324a28308ed2586472a. Latest consensus state: current_term: 1 leader_uuid: "6fce9948443c4324a28308ed2586472a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } } }
I20260619 02:30:04.820499 25843 tablet_replica.cc:439] T 3709c4bf47fe447198a270c8a07def72 P d1d857a119474703901ea085386d00c5: TxnStatusTablet state changed. Reason: New leader 6fce9948443c4324a28308ed2586472a. Latest consensus state: current_term: 1 leader_uuid: "6fce9948443c4324a28308ed2586472a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } } }
I20260619 02:30:04.820648 25868 tablet_replica.cc:442] T 3709c4bf47fe447198a270c8a07def72 P 393b9d3c902f4989b620d5db1cc80e01: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:30:04.820660 25843 tablet_replica.cc:442] T 3709c4bf47fe447198a270c8a07def72 P d1d857a119474703901ea085386d00c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:30:04.823807 25855 tablet_replica.cc:439] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6fce9948443c4324a28308ed2586472a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } } }
I20260619 02:30:04.823993 25855 tablet_replica.cc:442] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:30:04.824113 25855 tablet_replica.cc:439] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6fce9948443c4324a28308ed2586472a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } } }
I20260619 02:30:04.824199 25855 tablet_replica.cc:442] T 3709c4bf47fe447198a270c8a07def72 P 6fce9948443c4324a28308ed2586472a: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:30:04.825947 25851 mvcc.cc:205] Tried to move back new op lower bound from 7298401094925803520 to 7298401094832537600. Current Snapshot: MvccSnapshot[applied={T|T < 7298401094925803520}]
I20260619 02:30:04.825994 25868 tablet_replica.cc:439] T 3709c4bf47fe447198a270c8a07def72 P 393b9d3c902f4989b620d5db1cc80e01: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6fce9948443c4324a28308ed2586472a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } } }
I20260619 02:30:04.826145 25868 tablet_replica.cc:442] T 3709c4bf47fe447198a270c8a07def72 P 393b9d3c902f4989b620d5db1cc80e01: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:30:04.827155 25843 tablet_replica.cc:439] T 3709c4bf47fe447198a270c8a07def72 P d1d857a119474703901ea085386d00c5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6fce9948443c4324a28308ed2586472a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } } }
I20260619 02:30:04.827339 25843 tablet_replica.cc:442] T 3709c4bf47fe447198a270c8a07def72 P d1d857a119474703901ea085386d00c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:30:04.838991 25836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.833639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33314 (local address 127.17.96.194:42291)
0619 02:30:04.833862 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:04.833869 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:04.833918 (+ 49us) server_negotiation.cc:407] Connection header received
0619 02:30:04.834167 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:04.834171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:04.834284 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:04.834426 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:30:04.834810 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.835369 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.836863 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.837266 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.837379 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.837733 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:30:04.838434 (+ 701us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:30:04.838555 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:04.838694 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 02:30:04.838763 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 02:30:04.855592 25886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.849393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:42093 (local address 127.17.96.194:47129)
0619 02:30:04.849980 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:04.849997 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:04.850132 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:04.850539 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:04.850543 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:04.850557 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:30:04.850717 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.850725 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.852245 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.852249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.853703 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.853719 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.854524 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.854527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.854614 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.854684 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:30:04.855225 (+ 541us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:30:04.855232 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 02:30:04.855318 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 02:30:04.855411 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":139,"threads_started":1}
I20260619 02:30:04.855639 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.849731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:47129 (local address 127.17.96.195:42093)
0619 02:30:04.850025 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:04.850032 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:04.850063 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:30:04.850264 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:04.850268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:04.850337 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:04.850449 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:30:04.850940 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.852018 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.853923 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.854343 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.854462 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.854924 (+ 462us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:30:04.855083 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:30:04.855205 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:04.855340 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 02:30:04.855438 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260619 02:30:04.865446 25483 raft_consensus.cc:1261] T d862bc488d4e4eeba4b40d939afdcd65 P d1d857a119474703901ea085386d00c5 [term 1 FOLLOWER]: Refusing update from remote peer 393b9d3c902f4989b620d5db1cc80e01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:30:04.865698 25763 raft_consensus.cc:1261] T d862bc488d4e4eeba4b40d939afdcd65 P 6fce9948443c4324a28308ed2586472a [term 1 FOLLOWER]: Refusing update from remote peer 393b9d3c902f4989b620d5db1cc80e01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:30:04.866559 25868 consensus_queue.cc:1035] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:30:04.866852 25845 consensus_queue.cc:1035] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:30:04.871918 25850 mvcc.cc:205] Tried to move back new op lower bound from 7298401095122112512 to 7298401094399029248. Current Snapshot: MvccSnapshot[applied={T|T < 7298401095122112512}]
I20260619 02:30:04.900065 25836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.894621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:35445 (local address 127.17.96.194:42291)
0619 02:30:04.894775 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:04.894781 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:04.895220 (+ 439us) server_negotiation.cc:407] Connection header received
0619 02:30:04.895407 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:04.895410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:04.895485 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:04.895579 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:30:04.895986 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.896860 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.898198 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.898788 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.898939 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.899355 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:30:04.899484 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:30:04.899567 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:04.899801 (+ 234us) server_negotiation.cc:299] Negotiation successful
0619 02:30:04.899882 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:30:04.900089 25890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:04.894484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:42291 (local address 127.17.96.195:35445)
0619 02:30:04.895085 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:04.895107 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:04.895256 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:04.895632 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:04.895635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:04.895647 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:30:04.895846 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.895855 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.896969 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.896971 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.898035 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:04.898049 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.898965 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.898970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:04.899081 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.899157 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:30:04.899585 (+ 428us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:30:04.899598 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 02:30:04.899743 (+ 145us) client_negotiation.cc:240] Negotiation successful
0619 02:30:04.899867 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":155,"threads_started":1}
I20260619 02:30:04.940630 25326 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:30:04.940855 25326 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:30:04.943825 25326 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272: Sending DeleteTablet for 3 replicas of tablet d862bc488d4e4eeba4b40d939afdcd65
I20260619 02:30:04.944667 25603 tablet_service.cc:1514] Processing DeleteTablet for tablet d862bc488d4e4eeba4b40d939afdcd65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:30:04 UTC) from {username='oryx', principal='oryx/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:33304
I20260619 02:30:04.944674 25463 tablet_service.cc:1514] Processing DeleteTablet for tablet d862bc488d4e4eeba4b40d939afdcd65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:30:04 UTC) from {username='oryx', principal='oryx/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:33948
I20260619 02:30:04.944912 25860 tablet_replica.cc:330] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01: stopping tablet replica
I20260619 02:30:04.944963 25862 tablet_replica.cc:330] T d862bc488d4e4eeba4b40d939afdcd65 P d1d857a119474703901ea085386d00c5: stopping tablet replica
I20260619 02:30:04.945061 25860 raft_consensus.cc:2229] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:30:04.945124 25862 raft_consensus.cc:2229] T d862bc488d4e4eeba4b40d939afdcd65 P d1d857a119474703901ea085386d00c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:30:04.945335 25862 raft_consensus.cc:2258] T d862bc488d4e4eeba4b40d939afdcd65 P d1d857a119474703901ea085386d00c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:30:04.945443 25743 tablet_service.cc:1514] Processing DeleteTablet for tablet d862bc488d4e4eeba4b40d939afdcd65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:30:04 UTC) from {username='oryx', principal='oryx/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:37476
I20260619 02:30:04.945469 25860 raft_consensus.cc:2258] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:30:04.945842 25861 tablet_replica.cc:330] T d862bc488d4e4eeba4b40d939afdcd65 P 6fce9948443c4324a28308ed2586472a: stopping tablet replica
I20260619 02:30:04.945993 25861 raft_consensus.cc:2229] T d862bc488d4e4eeba4b40d939afdcd65 P 6fce9948443c4324a28308ed2586472a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:30:04.946141 25861 raft_consensus.cc:2258] T d862bc488d4e4eeba4b40d939afdcd65 P 6fce9948443c4324a28308ed2586472a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:30:04.947947 25861 ts_tablet_manager.cc:1905] T d862bc488d4e4eeba4b40d939afdcd65 P 6fce9948443c4324a28308ed2586472a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:30:04.952520 25861 ts_tablet_manager.cc:1918] T d862bc488d4e4eeba4b40d939afdcd65 P 6fce9948443c4324a28308ed2586472a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:30:04.952629 25861 log.cc:1196] T d862bc488d4e4eeba4b40d939afdcd65 P 6fce9948443c4324a28308ed2586472a: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/d862bc488d4e4eeba4b40d939afdcd65
I20260619 02:30:04.952947 25861 ts_tablet_manager.cc:1939] T d862bc488d4e4eeba4b40d939afdcd65 P 6fce9948443c4324a28308ed2586472a: Deleting consensus metadata
I20260619 02:30:04.954216 25860 ts_tablet_manager.cc:1905] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:30:04.959327 25862 ts_tablet_manager.cc:1905] T d862bc488d4e4eeba4b40d939afdcd65 P d1d857a119474703901ea085386d00c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:30:04.962066 25860 ts_tablet_manager.cc:1918] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:30:04.962203 25860 log.cc:1196] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/d862bc488d4e4eeba4b40d939afdcd65
I20260619 02:30:04.962558 25860 ts_tablet_manager.cc:1939] T d862bc488d4e4eeba4b40d939afdcd65 P 393b9d3c902f4989b620d5db1cc80e01: Deleting consensus metadata
I20260619 02:30:04.963554 25862 ts_tablet_manager.cc:1918] T d862bc488d4e4eeba4b40d939afdcd65 P d1d857a119474703901ea085386d00c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:30:04.963768 25862 log.cc:1196] T d862bc488d4e4eeba4b40d939afdcd65 P d1d857a119474703901ea085386d00c5: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-0/wal/wals/d862bc488d4e4eeba4b40d939afdcd65
I20260619 02:30:04.963724 25311 catalog_manager.cc:4841] TS 393b9d3c902f4989b620d5db1cc80e01 (127.17.96.194:42291): tablet d862bc488d4e4eeba4b40d939afdcd65 (table test-table [id=54607fd398b343348496da651e5de976]) successfully deleted
I20260619 02:30:04.964238 25862 ts_tablet_manager.cc:1939] T d862bc488d4e4eeba4b40d939afdcd65 P d1d857a119474703901ea085386d00c5: Deleting consensus metadata
I20260619 02:30:04.964787 25310 catalog_manager.cc:4841] TS 6fce9948443c4324a28308ed2586472a (127.17.96.195:42093): tablet d862bc488d4e4eeba4b40d939afdcd65 (table test-table [id=54607fd398b343348496da651e5de976]) successfully deleted
I20260619 02:30:04.965405 25310 catalog_manager.cc:4841] TS d1d857a119474703901ea085386d00c5 (127.17.96.193:37233): tablet d862bc488d4e4eeba4b40d939afdcd65 (table test-table [id=54607fd398b343348496da651e5de976]) successfully deleted
Jun 19 02:30:04 dist-test-slave-4qw6 krb5kdc[25281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:30:04 dist-test-slave-4qw6 krb5kdc[25281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:04 dist-test-slave-4qw6 krb5kdc[25281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
W20260619 02:30:04.996402 25375 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:04.984817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41386 (local address 127.17.96.254:37977)
0619 02:30:04.984970 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:04.984975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:04.985447 (+ 472us) server_negotiation.cc:407] Connection header received
0619 02:30:04.985591 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:04.985596 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:04.985668 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:04.985959 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:04.987603 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.988509 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.990472 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:04.990908 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:04.991096 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:04.996245 (+ 5149us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41386: BlockingRecv error: recv got EOF from 127.0.0.1:41386 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Jun 19 02:30:05 dist-test-slave-4qw6 krb5kdc[25281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM
I20260619 02:30:05.014992 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:05.000190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41390 (local address 127.17.96.254:37977)
0619 02:30:05.000356 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:05.000362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:05.001663 (+ 1301us) server_negotiation.cc:407] Connection header received
0619 02:30:05.001955 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:05.001960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:05.002027 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:05.002100 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:05.003502 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:05.004464 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:05.005979 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:05.006362 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:05.006459 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:05.010448 (+ 3989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:05.010454 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:05.010457 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:30:05.010512 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:05.012470 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:05.013072 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:05.013076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:05.013078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:05.013205 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:05.013615 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:05.013618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:05.013620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:05.013943 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:05.014079 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:05.014596 (+ 517us) server_negotiation.cc:299] Negotiation successful
0619 02:30:05.014771 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:30:05.019618 25326 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:30:05.020325 25326 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:30:05.027055 25463 tablet_service.cc:1467] Processing CreateTablet for tablet 9a47ba22d062474681fecbe35fc5f82e (DEFAULT_TABLE table=test-table [id=90fedc7ef81145258a87f113017956ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:30:05.027209 25603 tablet_service.cc:1467] Processing CreateTablet for tablet 9a47ba22d062474681fecbe35fc5f82e (DEFAULT_TABLE table=test-table [id=90fedc7ef81145258a87f113017956ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:30:05.027504 25463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a47ba22d062474681fecbe35fc5f82e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:05.027601 25603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a47ba22d062474681fecbe35fc5f82e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:05.027846 25743 tablet_service.cc:1467] Processing CreateTablet for tablet 9a47ba22d062474681fecbe35fc5f82e (DEFAULT_TABLE table=test-table [id=90fedc7ef81145258a87f113017956ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:30:05.028141 25743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a47ba22d062474681fecbe35fc5f82e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:05.030563 25838 tablet_bootstrap.cc:492] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01: Bootstrap starting.
I20260619 02:30:05.031319 25839 tablet_bootstrap.cc:492] T 9a47ba22d062474681fecbe35fc5f82e P 6fce9948443c4324a28308ed2586472a: Bootstrap starting.
I20260619 02:30:05.031744 25837 tablet_bootstrap.cc:492] T 9a47ba22d062474681fecbe35fc5f82e P d1d857a119474703901ea085386d00c5: Bootstrap starting.
I20260619 02:30:05.031754 25838 tablet_bootstrap.cc:654] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:05.032368 25839 tablet_bootstrap.cc:654] T 9a47ba22d062474681fecbe35fc5f82e P 6fce9948443c4324a28308ed2586472a: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:05.032927 25837 tablet_bootstrap.cc:654] T 9a47ba22d062474681fecbe35fc5f82e P d1d857a119474703901ea085386d00c5: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:05.034052 25839 tablet_bootstrap.cc:492] T 9a47ba22d062474681fecbe35fc5f82e P 6fce9948443c4324a28308ed2586472a: No bootstrap required, opened a new log
I20260619 02:30:05.034160 25839 ts_tablet_manager.cc:1397] T 9a47ba22d062474681fecbe35fc5f82e P 6fce9948443c4324a28308ed2586472a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 02:30:05.034247 25838 tablet_bootstrap.cc:492] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01: No bootstrap required, opened a new log
I20260619 02:30:05.034323 25838 ts_tablet_manager.cc:1397] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260619 02:30:05.034669 25839 raft_consensus.cc:348] T 9a47ba22d062474681fecbe35fc5f82e P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:05.034798 25839 raft_consensus.cc:374] T 9a47ba22d062474681fecbe35fc5f82e P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:05.034837 25839 raft_consensus.cc:729] T 9a47ba22d062474681fecbe35fc5f82e P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fce9948443c4324a28308ed2586472a, State: Initialized, Role: FOLLOWER
I20260619 02:30:05.034790 25838 raft_consensus.cc:348] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:05.034909 25838 raft_consensus.cc:374] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:05.034943 25838 raft_consensus.cc:729] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 393b9d3c902f4989b620d5db1cc80e01, State: Initialized, Role: FOLLOWER
I20260619 02:30:05.035000 25839 consensus_queue.cc:260] T 9a47ba22d062474681fecbe35fc5f82e P 6fce9948443c4324a28308ed2586472a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:05.035092 25838 consensus_queue.cc:260] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:05.035315 25839 ts_tablet_manager.cc:1428] T 9a47ba22d062474681fecbe35fc5f82e P 6fce9948443c4324a28308ed2586472a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 02:30:05.035369 25838 ts_tablet_manager.cc:1428] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:30:05.037246 25837 tablet_bootstrap.cc:492] T 9a47ba22d062474681fecbe35fc5f82e P d1d857a119474703901ea085386d00c5: No bootstrap required, opened a new log
I20260619 02:30:05.037379 25837 ts_tablet_manager.cc:1397] T 9a47ba22d062474681fecbe35fc5f82e P d1d857a119474703901ea085386d00c5: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260619 02:30:05.038067 25837 raft_consensus.cc:348] T 9a47ba22d062474681fecbe35fc5f82e P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:05.038239 25837 raft_consensus.cc:374] T 9a47ba22d062474681fecbe35fc5f82e P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:05.038282 25837 raft_consensus.cc:729] T 9a47ba22d062474681fecbe35fc5f82e P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1d857a119474703901ea085386d00c5, State: Initialized, Role: FOLLOWER
I20260619 02:30:05.038431 25837 consensus_queue.cc:260] T 9a47ba22d062474681fecbe35fc5f82e P d1d857a119474703901ea085386d00c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:05.038785 25837 ts_tablet_manager.cc:1428] T 9a47ba22d062474681fecbe35fc5f82e P d1d857a119474703901ea085386d00c5: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260619 02:30:05.125962 25845 raft_consensus.cc:482] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:30:05.126158 25845 raft_consensus.cc:504] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:05.126713 25845 leader_election.cc:288] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1d857a119474703901ea085386d00c5 (127.17.96.193:37233), 6fce9948443c4324a28308ed2586472a (127.17.96.195:42093)
I20260619 02:30:05.127311 25482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a47ba22d062474681fecbe35fc5f82e" candidate_uuid: "393b9d3c902f4989b620d5db1cc80e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1d857a119474703901ea085386d00c5" is_pre_election: true
I20260619 02:30:05.127357 25762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a47ba22d062474681fecbe35fc5f82e" candidate_uuid: "393b9d3c902f4989b620d5db1cc80e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fce9948443c4324a28308ed2586472a" is_pre_election: true
I20260619 02:30:05.127516 25762 raft_consensus.cc:2454] T 9a47ba22d062474681fecbe35fc5f82e P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 393b9d3c902f4989b620d5db1cc80e01 in term 0.
I20260619 02:30:05.127516 25482 raft_consensus.cc:2454] T 9a47ba22d062474681fecbe35fc5f82e P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 393b9d3c902f4989b620d5db1cc80e01 in term 0.
I20260619 02:30:05.127853 25554 leader_election.cc:302] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 393b9d3c902f4989b620d5db1cc80e01, d1d857a119474703901ea085386d00c5; no voters:
I20260619 02:30:05.128108 25845 raft_consensus.cc:2781] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:30:05.128181 25845 raft_consensus.cc:482] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:30:05.128247 25845 raft_consensus.cc:3037] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:05.129483 25845 raft_consensus.cc:504] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:05.130033 25845 leader_election.cc:288] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01 [CANDIDATE]: Term 1 election: Requested vote from peers d1d857a119474703901ea085386d00c5 (127.17.96.193:37233), 6fce9948443c4324a28308ed2586472a (127.17.96.195:42093)
I20260619 02:30:05.130528 25762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a47ba22d062474681fecbe35fc5f82e" candidate_uuid: "393b9d3c902f4989b620d5db1cc80e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fce9948443c4324a28308ed2586472a"
I20260619 02:30:05.130528 25482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a47ba22d062474681fecbe35fc5f82e" candidate_uuid: "393b9d3c902f4989b620d5db1cc80e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1d857a119474703901ea085386d00c5"
I20260619 02:30:05.130697 25482 raft_consensus.cc:3037] T 9a47ba22d062474681fecbe35fc5f82e P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:05.130707 25762 raft_consensus.cc:3037] T 9a47ba22d062474681fecbe35fc5f82e P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:05.132016 25762 raft_consensus.cc:2454] T 9a47ba22d062474681fecbe35fc5f82e P 6fce9948443c4324a28308ed2586472a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 393b9d3c902f4989b620d5db1cc80e01 in term 1.
I20260619 02:30:05.132503 25554 leader_election.cc:302] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 393b9d3c902f4989b620d5db1cc80e01, 6fce9948443c4324a28308ed2586472a; no voters:
I20260619 02:30:05.132756 25845 raft_consensus.cc:2781] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:30:05.132798 25482 raft_consensus.cc:2454] T 9a47ba22d062474681fecbe35fc5f82e P d1d857a119474703901ea085386d00c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 393b9d3c902f4989b620d5db1cc80e01 in term 1.
I20260619 02:30:05.132884 25845 raft_consensus.cc:686] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01 [term 1 LEADER]: Becoming Leader. State: Replica: 393b9d3c902f4989b620d5db1cc80e01, State: Running, Role: LEADER
I20260619 02:30:05.133075 25845 consensus_queue.cc:237] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:05.135502 25324 catalog_manager.cc:5495] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 393b9d3c902f4989b620d5db1cc80e01 (127.17.96.194). New cstate: current_term: 1 leader_uuid: "393b9d3c902f4989b620d5db1cc80e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } health_report { overall_health: UNKNOWN } } }
I20260619 02:30:05.168511 25836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:05.164942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33326 (local address 127.17.96.194:42291)
0619 02:30:05.165069 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:05.165073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:05.165120 (+ 47us) server_negotiation.cc:407] Connection header received
0619 02:30:05.165253 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:05.165255 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:05.165307 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:05.165378 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:30:05.165617 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:05.166168 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:05.167257 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:05.167490 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:05.167630 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:05.167983 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:30:05.168073 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:30:05.168162 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:05.168259 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 02:30:05.168317 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 02:30:05.170949 25762 raft_consensus.cc:1261] T 9a47ba22d062474681fecbe35fc5f82e P 6fce9948443c4324a28308ed2586472a [term 1 FOLLOWER]: Refusing update from remote peer 393b9d3c902f4989b620d5db1cc80e01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:30:05.171037 25482 raft_consensus.cc:1261] T 9a47ba22d062474681fecbe35fc5f82e P d1d857a119474703901ea085386d00c5 [term 1 FOLLOWER]: Refusing update from remote peer 393b9d3c902f4989b620d5db1cc80e01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:30:05.171504 25868 consensus_queue.cc:1035] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:30:05.171676 25845 consensus_queue.cc:1035] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:30:05.182139 25324 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:30:05.182309 25324 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:30:05.185256 25324 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272: Sending DeleteTablet for 3 replicas of tablet 9a47ba22d062474681fecbe35fc5f82e
I20260619 02:30:05.186148 25603 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a47ba22d062474681fecbe35fc5f82e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:30:05 UTC) from {username='oryx', principal='oryx/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:33304
I20260619 02:30:05.186271 25463 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a47ba22d062474681fecbe35fc5f82e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:30:05 UTC) from {username='oryx', principal='oryx/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:33948
I20260619 02:30:05.186260 25743 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a47ba22d062474681fecbe35fc5f82e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:30:05 UTC) from {username='oryx', principal='oryx/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:37476
I20260619 02:30:05.186441 25860 tablet_replica.cc:330] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01: stopping tablet replica
I20260619 02:30:05.186465 25862 tablet_replica.cc:330] T 9a47ba22d062474681fecbe35fc5f82e P d1d857a119474703901ea085386d00c5: stopping tablet replica
I20260619 02:30:05.186502 25861 tablet_replica.cc:330] T 9a47ba22d062474681fecbe35fc5f82e P 6fce9948443c4324a28308ed2586472a: stopping tablet replica
I20260619 02:30:05.186569 25862 raft_consensus.cc:2229] T 9a47ba22d062474681fecbe35fc5f82e P d1d857a119474703901ea085386d00c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:30:05.186587 25860 raft_consensus.cc:2229] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:30:05.186620 25861 raft_consensus.cc:2229] T 9a47ba22d062474681fecbe35fc5f82e P 6fce9948443c4324a28308ed2586472a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:30:05.186692 25862 raft_consensus.cc:2258] T 9a47ba22d062474681fecbe35fc5f82e P d1d857a119474703901ea085386d00c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:30:05.186787 25861 raft_consensus.cc:2258] T 9a47ba22d062474681fecbe35fc5f82e P 6fce9948443c4324a28308ed2586472a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:30:05.186823 25860 raft_consensus.cc:2258] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:30:05.187484 25862 ts_tablet_manager.cc:1905] T 9a47ba22d062474681fecbe35fc5f82e P d1d857a119474703901ea085386d00c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:30:05.187647 25861 ts_tablet_manager.cc:1905] T 9a47ba22d062474681fecbe35fc5f82e P 6fce9948443c4324a28308ed2586472a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:30:05.188251 25324 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 02:30:05.188453 25860 ts_tablet_manager.cc:1905] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 02:30:05.189088 25324 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:30:05.190344 25861 ts_tablet_manager.cc:1918] T 9a47ba22d062474681fecbe35fc5f82e P 6fce9948443c4324a28308ed2586472a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:30:05.190367 25862 ts_tablet_manager.cc:1918] T 9a47ba22d062474681fecbe35fc5f82e P d1d857a119474703901ea085386d00c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:30:05.190440 25862 log.cc:1196] T 9a47ba22d062474681fecbe35fc5f82e P d1d857a119474703901ea085386d00c5: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-0/wal/wals/9a47ba22d062474681fecbe35fc5f82e
I20260619 02:30:05.190440 25861 log.cc:1196] T 9a47ba22d062474681fecbe35fc5f82e P 6fce9948443c4324a28308ed2586472a: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/9a47ba22d062474681fecbe35fc5f82e
I20260619 02:30:05.190727 25862 ts_tablet_manager.cc:1939] T 9a47ba22d062474681fecbe35fc5f82e P d1d857a119474703901ea085386d00c5: Deleting consensus metadata
I20260619 02:30:05.190744 25861 ts_tablet_manager.cc:1939] T 9a47ba22d062474681fecbe35fc5f82e P 6fce9948443c4324a28308ed2586472a: Deleting consensus metadata
I20260619 02:30:05.191561 25310 catalog_manager.cc:4841] TS d1d857a119474703901ea085386d00c5 (127.17.96.193:37233): tablet 9a47ba22d062474681fecbe35fc5f82e (table test-table [id=90fedc7ef81145258a87f113017956ae]) successfully deleted
I20260619 02:30:05.191751 25310 catalog_manager.cc:4841] TS 6fce9948443c4324a28308ed2586472a (127.17.96.195:42093): tablet 9a47ba22d062474681fecbe35fc5f82e (table test-table [id=90fedc7ef81145258a87f113017956ae]) successfully deleted
I20260619 02:30:05.191751 25860 ts_tablet_manager.cc:1918] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:30:05.191859 25860 log.cc:1196] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/9a47ba22d062474681fecbe35fc5f82e
I20260619 02:30:05.192174 25860 ts_tablet_manager.cc:1939] T 9a47ba22d062474681fecbe35fc5f82e P 393b9d3c902f4989b620d5db1cc80e01: Deleting consensus metadata
I20260619 02:30:05.193104 25311 catalog_manager.cc:4841] TS 393b9d3c902f4989b620d5db1cc80e01 (127.17.96.194:42291): tablet 9a47ba22d062474681fecbe35fc5f82e (table test-table [id=90fedc7ef81145258a87f113017956ae]) successfully deleted
I20260619 02:30:05.195815 25463 tablet_service.cc:1467] Processing CreateTablet for tablet d89b20e053fe46fd9614a925d7628e8c (DEFAULT_TABLE table=test-table [id=20c547ca3de5461c95f3d3ed016f2233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:30:05.195951 25603 tablet_service.cc:1467] Processing CreateTablet for tablet d89b20e053fe46fd9614a925d7628e8c (DEFAULT_TABLE table=test-table [id=20c547ca3de5461c95f3d3ed016f2233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:30:05.196045 25743 tablet_service.cc:1467] Processing CreateTablet for tablet d89b20e053fe46fd9614a925d7628e8c (DEFAULT_TABLE table=test-table [id=20c547ca3de5461c95f3d3ed016f2233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:30:05.196192 25463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d89b20e053fe46fd9614a925d7628e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:05.196305 25603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d89b20e053fe46fd9614a925d7628e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:05.196342 25743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d89b20e053fe46fd9614a925d7628e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:05.198954 25837 tablet_bootstrap.cc:492] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5: Bootstrap starting.
I20260619 02:30:05.199463 25839 tablet_bootstrap.cc:492] T d89b20e053fe46fd9614a925d7628e8c P 6fce9948443c4324a28308ed2586472a: Bootstrap starting.
I20260619 02:30:05.200006 25837 tablet_bootstrap.cc:654] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:05.200310 25839 tablet_bootstrap.cc:654] T d89b20e053fe46fd9614a925d7628e8c P 6fce9948443c4324a28308ed2586472a: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:05.201212 25838 tablet_bootstrap.cc:492] T d89b20e053fe46fd9614a925d7628e8c P 393b9d3c902f4989b620d5db1cc80e01: Bootstrap starting.
I20260619 02:30:05.201366 25837 tablet_bootstrap.cc:492] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5: No bootstrap required, opened a new log
I20260619 02:30:05.201465 25837 ts_tablet_manager.cc:1397] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 02:30:05.202083 25837 raft_consensus.cc:348] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:05.202230 25837 raft_consensus.cc:374] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:05.202271 25839 tablet_bootstrap.cc:492] T d89b20e053fe46fd9614a925d7628e8c P 6fce9948443c4324a28308ed2586472a: No bootstrap required, opened a new log
I20260619 02:30:05.202270 25837 raft_consensus.cc:729] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1d857a119474703901ea085386d00c5, State: Initialized, Role: FOLLOWER
I20260619 02:30:05.202366 25839 ts_tablet_manager.cc:1397] T d89b20e053fe46fd9614a925d7628e8c P 6fce9948443c4324a28308ed2586472a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 02:30:05.202368 25838 tablet_bootstrap.cc:654] T d89b20e053fe46fd9614a925d7628e8c P 393b9d3c902f4989b620d5db1cc80e01: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:05.202452 25837 consensus_queue.cc:260] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:05.202869 25837 ts_tablet_manager.cc:1428] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260619 02:30:05.202831 25839 raft_consensus.cc:348] T d89b20e053fe46fd9614a925d7628e8c P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:05.202950 25839 raft_consensus.cc:374] T d89b20e053fe46fd9614a925d7628e8c P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:05.203007 25839 raft_consensus.cc:729] T d89b20e053fe46fd9614a925d7628e8c P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fce9948443c4324a28308ed2586472a, State: Initialized, Role: FOLLOWER
I20260619 02:30:05.203138 25839 consensus_queue.cc:260] T d89b20e053fe46fd9614a925d7628e8c P 6fce9948443c4324a28308ed2586472a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:05.203446 25839 ts_tablet_manager.cc:1428] T d89b20e053fe46fd9614a925d7628e8c P 6fce9948443c4324a28308ed2586472a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:30:05.203898 25838 tablet_bootstrap.cc:492] T d89b20e053fe46fd9614a925d7628e8c P 393b9d3c902f4989b620d5db1cc80e01: No bootstrap required, opened a new log
I20260619 02:30:05.204000 25838 ts_tablet_manager.cc:1397] T d89b20e053fe46fd9614a925d7628e8c P 393b9d3c902f4989b620d5db1cc80e01: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 02:30:05.204387 25838 raft_consensus.cc:348] T d89b20e053fe46fd9614a925d7628e8c P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:05.204537 25838 raft_consensus.cc:374] T d89b20e053fe46fd9614a925d7628e8c P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:05.204576 25838 raft_consensus.cc:729] T d89b20e053fe46fd9614a925d7628e8c P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 393b9d3c902f4989b620d5db1cc80e01, State: Initialized, Role: FOLLOWER
I20260619 02:30:05.204704 25838 consensus_queue.cc:260] T d89b20e053fe46fd9614a925d7628e8c P 393b9d3c902f4989b620d5db1cc80e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:05.204989 25838 ts_tablet_manager.cc:1428] T d89b20e053fe46fd9614a925d7628e8c P 393b9d3c902f4989b620d5db1cc80e01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:30:05.350319 25843 raft_consensus.cc:482] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:30:05.350486 25843 raft_consensus.cc:504] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:05.351796 25843 leader_election.cc:288] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 393b9d3c902f4989b620d5db1cc80e01 (127.17.96.194:42291), 6fce9948443c4324a28308ed2586472a (127.17.96.195:42093)
I20260619 02:30:05.356707 25917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:05.352077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:42291 (local address 127.17.96.193:33079)
0619 02:30:05.352444 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:05.352458 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:05.352675 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:05.353056 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:05.353059 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:05.353076 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:30:05.353196 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:05.353201 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:05.354267 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:05.354270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:05.355211 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:05.355221 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:05.355985 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:05.355989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:05.356171 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:05.356205 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 02:30:05.356395 (+ 190us) client_negotiation.cc:240] Negotiation successful
0619 02:30:05.356496 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":41,"thread_start_us":96,"threads_started":1}
I20260619 02:30:05.356735 25836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:05.352195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:33079 (local address 127.17.96.194:42291)
0619 02:30:05.352347 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:05.352352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:05.352548 (+ 196us) server_negotiation.cc:407] Connection header received
0619 02:30:05.352851 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:05.352854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:05.352935 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:05.353021 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:30:05.353374 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:05.354117 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:05.355367 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:05.355843 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:05.355995 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:05.356055 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:05.356420 (+ 365us) server_negotiation.cc:299] Negotiation successful
0619 02:30:05.356558 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 02:30:05.356909 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:05.352235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:37439 (local address 127.17.96.195:42093)
0619 02:30:05.352387 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:05.352393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:05.352645 (+ 252us) server_negotiation.cc:407] Connection header received
0619 02:30:05.352802 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:05.352805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:05.352858 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:05.352936 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:30:05.353280 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:05.354023 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:05.355343 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:05.355785 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:05.355947 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:05.356011 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:05.356596 (+ 585us) server_negotiation.cc:299] Negotiation successful
0619 02:30:05.356720 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 02:30:05.357308 25623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d89b20e053fe46fd9614a925d7628e8c" candidate_uuid: "d1d857a119474703901ea085386d00c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "393b9d3c902f4989b620d5db1cc80e01" is_pre_election: true
I20260619 02:30:05.357442 25916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:05.352158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:42093 (local address 127.17.96.193:37439)
0619 02:30:05.352522 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:05.352535 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:05.352683 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:05.352982 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:05.352986 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:05.353010 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:30:05.353165 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:05.353172 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:05.354144 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:05.354148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:05.355201 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:05.355212 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:05.355948 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:05.355952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:05.356393 (+ 441us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:05.356434 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 02:30:05.357171 (+ 737us) client_negotiation.cc:240] Negotiation successful
0619 02:30:05.357296 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":4,"thread_start_us":85,"threads_started":1}
I20260619 02:30:05.357512 25623 raft_consensus.cc:2454] T d89b20e053fe46fd9614a925d7628e8c P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1d857a119474703901ea085386d00c5 in term 0.
I20260619 02:30:05.358019 25762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d89b20e053fe46fd9614a925d7628e8c" candidate_uuid: "d1d857a119474703901ea085386d00c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fce9948443c4324a28308ed2586472a" is_pre_election: true
I20260619 02:30:05.358209 25762 raft_consensus.cc:2454] T d89b20e053fe46fd9614a925d7628e8c P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1d857a119474703901ea085386d00c5 in term 0.
I20260619 02:30:05.358223 25415 leader_election.cc:302] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 393b9d3c902f4989b620d5db1cc80e01, d1d857a119474703901ea085386d00c5; no voters:
I20260619 02:30:05.358567 25843 raft_consensus.cc:2781] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:30:05.358628 25843 raft_consensus.cc:482] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:30:05.358659 25843 raft_consensus.cc:3037] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:05.359800 25843 raft_consensus.cc:504] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:05.360225 25843 leader_election.cc:288] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5 [CANDIDATE]: Term 1 election: Requested vote from peers 393b9d3c902f4989b620d5db1cc80e01 (127.17.96.194:42291), 6fce9948443c4324a28308ed2586472a (127.17.96.195:42093)
I20260619 02:30:05.360653 25762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d89b20e053fe46fd9614a925d7628e8c" candidate_uuid: "d1d857a119474703901ea085386d00c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fce9948443c4324a28308ed2586472a"
I20260619 02:30:05.360846 25762 raft_consensus.cc:3037] T d89b20e053fe46fd9614a925d7628e8c P 6fce9948443c4324a28308ed2586472a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:05.360812 25623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d89b20e053fe46fd9614a925d7628e8c" candidate_uuid: "d1d857a119474703901ea085386d00c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "393b9d3c902f4989b620d5db1cc80e01"
I20260619 02:30:05.360954 25623 raft_consensus.cc:3037] T d89b20e053fe46fd9614a925d7628e8c P 393b9d3c902f4989b620d5db1cc80e01 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:05.362283 25762 raft_consensus.cc:2454] T d89b20e053fe46fd9614a925d7628e8c P 6fce9948443c4324a28308ed2586472a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1d857a119474703901ea085386d00c5 in term 1.
I20260619 02:30:05.362289 25623 raft_consensus.cc:2454] T d89b20e053fe46fd9614a925d7628e8c P 393b9d3c902f4989b620d5db1cc80e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1d857a119474703901ea085386d00c5 in term 1.
I20260619 02:30:05.362713 25415 leader_election.cc:302] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 393b9d3c902f4989b620d5db1cc80e01, 6fce9948443c4324a28308ed2586472a, d1d857a119474703901ea085386d00c5; no voters:
I20260619 02:30:05.362951 25843 raft_consensus.cc:2781] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:30:05.363197 25843 raft_consensus.cc:686] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5 [term 1 LEADER]: Becoming Leader. State: Replica: d1d857a119474703901ea085386d00c5, State: Running, Role: LEADER
I20260619 02:30:05.363562 25843 consensus_queue.cc:237] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } }
I20260619 02:30:05.366393 25326 catalog_manager.cc:5495] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1d857a119474703901ea085386d00c5 (127.17.96.193). New cstate: current_term: 1 leader_uuid: "d1d857a119474703901ea085386d00c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d857a119474703901ea085386d00c5" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 37233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 } health_report { overall_health: UNKNOWN } } }
I20260619 02:30:05.418180 25920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:05.413104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33958 (local address 127.17.96.193:37233)
0619 02:30:05.413611 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:05.413621 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:05.413635 (+ 14us) server_negotiation.cc:407] Connection header received
0619 02:30:05.413728 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:05.413731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:05.413830 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:05.413965 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:30:05.414267 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:05.415184 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:05.416148 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:05.416576 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:05.416688 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:05.416974 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:30:05.417633 (+ 659us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:30:05.417790 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:05.417925 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 02:30:05.417994 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":165,"threads_started":1}
I20260619 02:30:05.430998 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:05.427277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:58695 (local address 127.17.96.195:42093)
0619 02:30:05.427455 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:05.427459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:05.427559 (+ 100us) server_negotiation.cc:407] Connection header received
0619 02:30:05.427696 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:05.427699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:05.427767 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:05.427849 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:30:05.428125 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:05.428703 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:05.429855 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:05.430116 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:05.430216 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:05.430560 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:30:05.430641 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:30:05.430707 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:05.430814 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 02:30:05.430867 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 02:30:05.431016 25923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:05.427073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:42093 (local address 127.17.96.193:58695)
0619 02:30:05.427462 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:05.427479 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:05.427592 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:05.427876 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:05.427878 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:05.427885 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:30:05.428024 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:05.428029 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:05.428912 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:05.428916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:05.429635 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:05.429645 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:05.430258 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:05.430261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:05.430374 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:05.430428 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:30:05.430727 (+ 299us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:30:05.430732 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:30:05.430788 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 02:30:05.430864 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":135,"threads_started":1}
I20260619 02:30:05.438900 25762 raft_consensus.cc:1261] T d89b20e053fe46fd9614a925d7628e8c P 6fce9948443c4324a28308ed2586472a [term 1 FOLLOWER]: Refusing update from remote peer d1d857a119474703901ea085386d00c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:30:05.438944 25623 raft_consensus.cc:1261] T d89b20e053fe46fd9614a925d7628e8c P 393b9d3c902f4989b620d5db1cc80e01 [term 1 FOLLOWER]: Refusing update from remote peer d1d857a119474703901ea085386d00c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:30:05.439529 25918 consensus_queue.cc:1035] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fce9948443c4324a28308ed2586472a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:30:05.439813 25843 consensus_queue.cc:1035] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "393b9d3c902f4989b620d5db1cc80e01" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 42291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:30:05.460482 25920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:05.456330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:43651 (local address 127.17.96.193:37233)
0619 02:30:05.456470 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:05.456475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:05.456950 (+ 475us) server_negotiation.cc:407] Connection header received
0619 02:30:05.457040 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:05.457048 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:05.457120 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:05.457203 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:30:05.457572 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:05.458233 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:05.459285 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:05.459545 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:05.459679 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:05.459964 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:30:05.460052 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:30:05.460147 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:05.460245 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 02:30:05.460306 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 02:30:05.460491 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:05.456220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:37233 (local address 127.17.96.195:43651)
0619 02:30:05.456579 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:05.456596 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:05.456844 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:05.457340 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:05.457342 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:05.457350 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:30:05.457446 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:05.457450 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:05.458346 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:05.458348 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:05.459142 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:05.459152 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:05.459683 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:05.459687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:05.459787 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:05.459819 (+ 32us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:30:05.460153 (+ 334us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:30:05.460162 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 02:30:05.460235 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 02:30:05.460345 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":106,"threads_started":1}
I20260619 02:30:05.494505 25326 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:30:05.494680 25326 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:30:05.497440 25326 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc018e2bb5d0430ea20129395ff03272: Sending DeleteTablet for 3 replicas of tablet d89b20e053fe46fd9614a925d7628e8c
I20260619 02:30:05.498498 25463 tablet_service.cc:1514] Processing DeleteTablet for tablet d89b20e053fe46fd9614a925d7628e8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:30:05 UTC) from {username='oryx', principal='oryx/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:33948
I20260619 02:30:05.498600 25743 tablet_service.cc:1514] Processing DeleteTablet for tablet d89b20e053fe46fd9614a925d7628e8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:30:05 UTC) from {username='oryx', principal='oryx/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:37476
I20260619 02:30:05.498754 25862 tablet_replica.cc:330] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5: stopping tablet replica
I20260619 02:30:05.498895 25862 raft_consensus.cc:2229] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:30:05.499140 25603 tablet_service.cc:1514] Processing DeleteTablet for tablet d89b20e053fe46fd9614a925d7628e8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:30:05 UTC) from {username='oryx', principal='oryx/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:33304
I20260619 02:30:05.499303 25862 raft_consensus.cc:2258] T d89b20e053fe46fd9614a925d7628e8c P d1d857a119474703901ea085386d00c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:30:05.499379 25860 tablet_replica.cc:330] T d89b20e053fe46fd9614a925d7628e8c P 393b9d3c902f4989b620d5db1cc80e01: stopping tablet replica
I20260619 02:30:05.499503 25861 tablet_replica.cc:330] T d89b20e053fe46fd9614a925d7628e8c P 6fce9948443c4324a28308ed2586472a: stopping tablet replica
I20260619 02:30:05.499511 25860 raft_consensus.cc:2229] T d89b20e053fe46fd9614a925d7628e8c P 393b9d3c902f4989b620d5db1cc80e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:30:05.499636 25860 raft_consensus.cc:2258] T d89b20e053fe46fd9614a925d7628e8c P 393b9d3c902f4989b620d5db1cc80e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:30:05.499634 25861 raft_consensus.cc:2229] T d89b20e053fe46fd9614a925d7628e8c P 6fce9948443c4324a28308ed2586472a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:30:05.499850 25861 raft_consensus.cc:2258] T d89b20e053fe46fd9614a925d7628e8c P 6fce9948443c4324a28308ed2586472a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:30:05.500677 25861 ts_tablet_manager.cc:1905] T d89b20e053fe46fd9614a925d7628e8c P 6fce9948443c4324a28308ed2586472a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:30:05.500677 25860 ts_tablet_manager.cc:1905] T d89b20e053fe46fd9614a925d7628e8c P 393b9d3c902f4989b620d5db1cc80e01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:30:05.500814 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 25398
I20260619 02:30:05.503899 25861 ts_tablet_manager.cc:1918] T d89b20e053fe46fd9614a925d7628e8c P 6fce9948443c4324a28308ed2586472a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:30:05.504021 25861 log.cc:1196] T d89b20e053fe46fd9614a925d7628e8c P 6fce9948443c4324a28308ed2586472a: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/d89b20e053fe46fd9614a925d7628e8c
I20260619 02:30:05.504169 25860 ts_tablet_manager.cc:1918] T d89b20e053fe46fd9614a925d7628e8c P 393b9d3c902f4989b620d5db1cc80e01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:30:05.504252 25860 log.cc:1196] T d89b20e053fe46fd9614a925d7628e8c P 393b9d3c902f4989b620d5db1cc80e01: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/d89b20e053fe46fd9614a925d7628e8c
I20260619 02:30:05.504354 25861 ts_tablet_manager.cc:1939] T d89b20e053fe46fd9614a925d7628e8c P 6fce9948443c4324a28308ed2586472a: Deleting consensus metadata
I20260619 02:30:05.504559 25860 ts_tablet_manager.cc:1939] T d89b20e053fe46fd9614a925d7628e8c P 393b9d3c902f4989b620d5db1cc80e01: Deleting consensus metadata
I20260619 02:30:05.505322 25310 catalog_manager.cc:4841] TS 6fce9948443c4324a28308ed2586472a (127.17.96.195:42093): tablet d89b20e053fe46fd9614a925d7628e8c (table test-table [id=20c547ca3de5461c95f3d3ed016f2233]) successfully deleted
I20260619 02:30:05.505479 25311 catalog_manager.cc:4841] TS 393b9d3c902f4989b620d5db1cc80e01 (127.17.96.194:42291): tablet d89b20e053fe46fd9614a925d7628e8c (table test-table [id=20c547ca3de5461c95f3d3ed016f2233]) successfully deleted
W20260619 02:30:05.512189 25310 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.96.193:37233 (error 108)
I20260619 02:30:05.512518 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 25538
I20260619 02:30:05.514819 25933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:05.513946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:37233 (local address 127.0.0.1:33974)
0619 02:30:05.514617 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:05.514705 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.193:37233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":79,"threads_started":1}
W20260619 02:30:05.514997 25310 catalog_manager.cc:4584] TS d1d857a119474703901ea085386d00c5 (127.17.96.193:37233): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d89b20e053fe46fd9614a925d7628e8c: Network error: Client connection negotiation failed: client connection to 127.17.96.193:37233: connect: Connection refused (error 111)
I20260619 02:30:05.522198 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 25678
I20260619 02:30:05.532536 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 25294
2026-06-19T02:30:05Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3659 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:30:05 dist-test-slave-4qw6 krb5kdc[25936](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:30:05 dist-test-slave-4qw6 krb5kdc[25936](info): set up 2 sockets
Jun 19 02:30:05 dist-test-slave-4qw6 krb5kdc[25936](info): commencing operation
krb5kdc: starting...
W20260619 02:30:06.671145 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:30:06 dist-test-slave-4qw6 krb5kdc[25936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:30:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:30:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:30:06.816586 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:34287
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:44811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:34287
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:30:06.945880 25949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:06.946221 25949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:06.946269 25949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:06.950929 25949 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:30:06.951018 25949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:06.951040 25949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:06.951059 25949 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:30:06.951077 25949 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:30:06.957506 25949 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:44811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:34287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:34287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:06.958922 25949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:06.960135 25949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:06.970201 25955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:06.970422 25949 server_base.cc:1034] running on GCE node
W20260619 02:30:06.970201 25954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:06.970657 25957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:06.971315 25949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:06.972596 25949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:06.973819 25949 hybrid_clock.cc:648] HybridClock initialized: now 1781836206973792 us; error 43 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-19T02:30:06Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1451 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:30:07 dist-test-slave-4qw6 krb5kdc[25962](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:30:07 dist-test-slave-4qw6 krb5kdc[25962](info): set up 2 sockets
Jun 19 02:30:07 dist-test-slave-4qw6 krb5kdc[25962](info): commencing operation
krb5kdc: starting...
W20260619 02:30:08.122881 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:30:08 dist-test-slave-4qw6 krb5kdc[25962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:30:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:30:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:30:08.268746 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:45519
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:45519
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:30:08.397475 25975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:08.397954 25975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:08.398054 25975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:08.402736 25975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:30:08.402897 25975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:08.402944 25975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:08.402997 25975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:30:08.403040 25975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:30:08.409447 25975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:45519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:45519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:08.410836 25975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:08.412047 25975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:08.422804 25981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:08.422967 25980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:08.423152 25983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:08.423389 25975 server_base.cc:1034] running on GCE node
I20260619 02:30:08.424069 25975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:08.425460 25975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:08.426663 25975 hybrid_clock.cc:648] HybridClock initialized: now 1781836208426641 us; error 44 us; skew 500 ppm
Jun 19 02:30:08 dist-test-slave-4qw6 krb5kdc[25962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836208, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:30:08.430150 25975 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:30:08.431399 25975 webserver.cc:460] Webserver started at http://127.17.96.254:34571/ using document root <none> and password file <none>
I20260619 02:30:08.432085 25975 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:08.432165 25975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:08.432367 25975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:30:08.434878 25975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "a14cf94de8564cb69a12440f5baf5483"
format_stamp: "Formatted at 2026-06-19 02:30:08 on dist-test-slave-4qw6"
server_key: "03dff33546e504a609a2f7ed159f02a8"
server_key_iv: "673faecda15692e8a8696a6fdb5c81e9"
server_key_version: "encryptionkey@0"
I20260619 02:30:08.435563 25975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "a14cf94de8564cb69a12440f5baf5483"
format_stamp: "Formatted at 2026-06-19 02:30:08 on dist-test-slave-4qw6"
server_key: "03dff33546e504a609a2f7ed159f02a8"
server_key_iv: "673faecda15692e8a8696a6fdb5c81e9"
server_key_version: "encryptionkey@0"
I20260619 02:30:08.440667 25975 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:30:08.444001 25990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:08.445529 25975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:30:08.445726 25975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "a14cf94de8564cb69a12440f5baf5483"
format_stamp: "Formatted at 2026-06-19 02:30:08 on dist-test-slave-4qw6"
server_key: "03dff33546e504a609a2f7ed159f02a8"
server_key_iv: "673faecda15692e8a8696a6fdb5c81e9"
server_key_version: "encryptionkey@0"
I20260619 02:30:08.445950 25975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:08.481124 25975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:08.485513 25975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:08.485723 25975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:08.493857 25975 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:45519
I20260619 02:30:08.493885 26042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:45519 every 8 connection(s)
I20260619 02:30:08.495024 25975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:30:08.498329 26043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:08.499037 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 25975
I20260619 02:30:08.499189 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:30:08.499536 17795 external_mini_cluster.cc:1428] Setting key 29f5d91f6ccf2e8c2388ddc73fb52882
I20260619 02:30:08.506517 26043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483: Bootstrap starting.
I20260619 02:30:08.509690 26043 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:08.510658 26043 log.cc:824] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483: Log is configured to *not* fsync() on all Append() calls
Jun 19 02:30:08 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:30:08.513451 26043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483: No bootstrap required, opened a new log
I20260619 02:30:08.518700 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:08.501180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58624 (local address 127.17.96.254:45519)
0619 02:30:08.501689 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:08.501710 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:08.501839 (+ 129us) server_negotiation.cc:407] Connection header received
0619 02:30:08.502756 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:08.502796 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:08.503289 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:08.503699 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:08.504582 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:08.506216 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:08.507328 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:08.508154 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:08.508370 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:08.511734 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:08.511741 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:08.511770 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:30:08.511802 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:08.514000 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:08.514508 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:08.514516 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:08.514523 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:08.514686 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:08.514976 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:08.514979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:08.514981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:08.515573 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:08.515828 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:08.516894 (+ 1066us) server_negotiation.cc:299] Negotiation successful
0619 02:30:08.517439 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":149,"threads_started":1}
I20260619 02:30:08.519397 26043 raft_consensus.cc:348] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14cf94de8564cb69a12440f5baf5483" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45519 } }
I20260619 02:30:08.519748 26043 raft_consensus.cc:374] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:08.519816 26043 raft_consensus.cc:729] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a14cf94de8564cb69a12440f5baf5483, State: Initialized, Role: FOLLOWER
I20260619 02:30:08.520380 26043 consensus_queue.cc:260] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14cf94de8564cb69a12440f5baf5483" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45519 } }
I20260619 02:30:08.520540 26043 raft_consensus.cc:388] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:30:08.520658 26043 raft_consensus.cc:482] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:30:08.520826 26043 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:08.522289 26043 raft_consensus.cc:504] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14cf94de8564cb69a12440f5baf5483" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45519 } }
I20260619 02:30:08.522760 26043 leader_election.cc:302] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a14cf94de8564cb69a12440f5baf5483; no voters:
I20260619 02:30:08.523206 26043 leader_election.cc:288] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:30:08.523590 26048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:30:08.523975 26048 raft_consensus.cc:686] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [term 1 LEADER]: Becoming Leader. State: Replica: a14cf94de8564cb69a12440f5baf5483, State: Running, Role: LEADER
I20260619 02:30:08.524494 26048 consensus_queue.cc:237] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14cf94de8564cb69a12440f5baf5483" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45519 } }
I20260619 02:30:08.524871 26043 sys_catalog.cc:564] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:30:08.526669 26050 sys_catalog.cc:455] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a14cf94de8564cb69a12440f5baf5483. Latest consensus state: current_term: 1 leader_uuid: "a14cf94de8564cb69a12440f5baf5483" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14cf94de8564cb69a12440f5baf5483" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45519 } } }
I20260619 02:30:08.526851 26050 sys_catalog.cc:458] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [sys.catalog]: This master's current role is: LEADER
I20260619 02:30:08.527226 26049 sys_catalog.cc:455] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a14cf94de8564cb69a12440f5baf5483" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14cf94de8564cb69a12440f5baf5483" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45519 } } }
I20260619 02:30:08.527339 26049 sys_catalog.cc:458] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [sys.catalog]: This master's current role is: LEADER
I20260619 02:30:08.527505 26058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:30:08.531100 26058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:30:08.538341 26058 catalog_manager.cc:1347] Generated new cluster ID: 5a85a589e0e9400791c32ce673dc5ec2
I20260619 02:30:08.538455 26058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:30:08.552062 26058 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:30:08.553550 26058 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:30:08.562214 26058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483: Generated new TSK 0
I20260619 02:30:08.563333 26058 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.96.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
I20260619 02:30:08.601972 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45519
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:30:08.745611 26069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:08.746068 26069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:08.746122 26069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:08.751235 26069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:08.751374 26069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:08.751497 26069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:30:08.759532 26069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:08.761330 26069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:08.762809 26069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:08.775712 26075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:08.775833 26069 server_base.cc:1034] running on GCE node
W20260619 02:30:08.775715 26074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:08.775712 26077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:08.776667 26069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:08.777439 26069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:08.778649 26069 hybrid_clock.cc:648] HybridClock initialized: now 1781836208778628 us; error 56 us; skew 500 ppm
Jun 19 02:30:08 dist-test-slave-4qw6 krb5kdc[25962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836208, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:30:08.782114 26069 init.cc:377] Logged in from keytab as kudu/127.17.96.193@KRBTEST.COM (short username kudu)
I20260619 02:30:08.783277 26069 webserver.cc:460] Webserver started at http://127.17.96.193:38089/ using document root <none> and password file <none>
I20260619 02:30:08.783985 26069 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:08.784096 26069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:08.784350 26069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:30:08.786906 26069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "46241c037aca4af99fb2015c12c4e268"
format_stamp: "Formatted at 2026-06-19 02:30:08 on dist-test-slave-4qw6"
server_key: "f0b6c1b8c1e7134a5bd6601cbf69bb91"
server_key_iv: "3dd1f825901a7472f91c7841cab1c381"
server_key_version: "encryptionkey@0"
I20260619 02:30:08.787619 26069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "46241c037aca4af99fb2015c12c4e268"
format_stamp: "Formatted at 2026-06-19 02:30:08 on dist-test-slave-4qw6"
server_key: "f0b6c1b8c1e7134a5bd6601cbf69bb91"
server_key_iv: "3dd1f825901a7472f91c7841cab1c381"
server_key_version: "encryptionkey@0"
I20260619 02:30:08.792806 26069 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:30:08.796317 26084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:08.798130 26069 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:30:08.798357 26069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "46241c037aca4af99fb2015c12c4e268"
format_stamp: "Formatted at 2026-06-19 02:30:08 on dist-test-slave-4qw6"
server_key: "f0b6c1b8c1e7134a5bd6601cbf69bb91"
server_key_iv: "3dd1f825901a7472f91c7841cab1c381"
server_key_version: "encryptionkey@0"
I20260619 02:30:08.798520 26069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:08.810454 26069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:08.814986 26069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:08.815315 26069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:08.816237 26069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:30:08.817507 26069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:30:08.817589 26069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:08.817687 26069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:30:08.817731 26069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:08.830394 26069 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:39975
I20260619 02:30:08.830513 26197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:39975 every 8 connection(s)
I20260619 02:30:08.831761 26069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:30:08.832628 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 26069
I20260619 02:30:08.832820 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:30:08.833191 17795 external_mini_cluster.cc:1428] Setting key da9ceb92ebcd396071fc4a36954391bb
WARNING: no policy specified for kudu/127.17.96.194@KRBTEST.COM; defaulting to no policy
Jun 19 02:30:08 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836208, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.194@KRBTEST.COM" created.
I20260619 02:30:08.854466 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:08.834567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:60671 (local address 127.17.96.254:45519)
0619 02:30:08.834759 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:08.834765 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:08.836300 (+ 1535us) server_negotiation.cc:407] Connection header received
0619 02:30:08.838271 (+ 1971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:08.838275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:08.838338 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:08.838425 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:08.841443 (+ 3018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:08.842214 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:08.844648 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:08.844943 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:08.845093 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:08.849172 (+ 4079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:08.849177 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:08.849179 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:30:08.849212 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:08.851389 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:08.852290 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:08.852293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:08.852295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:08.852380 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:08.852859 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:08.852861 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:08.852863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:08.853175 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:08.853317 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:08.854023 (+ 706us) server_negotiation.cc:299] Negotiation successful
0619 02:30:08.854213 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 02:30:08.855391 26200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:08.835354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.193:60671)
0619 02:30:08.836095 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:08.836158 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:08.837606 (+ 1448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:08.839057 (+ 1451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:08.839073 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:08.840118 (+ 1045us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:08.841128 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:08.841162 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:08.842486 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:08.842492 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:08.844418 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:08.844441 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:08.845199 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:08.845207 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:08.845513 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:08.846594 (+ 1081us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:08.846634 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:08.848884 (+ 2250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:30:08.851580 (+ 2696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:08.851592 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:08.851619 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:08.852149 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:08.852489 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:08.852495 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:08.852500 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:08.852746 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:08.853311 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:30:08.853319 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:30:08.853657 (+ 338us) client_negotiation.cc:769] Sending connection context
0619 02:30:08.853935 (+ 278us) client_negotiation.cc:240] Negotiation successful
0619 02:30:08.854293 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":200,"threads_started":1}
I20260619 02:30:08.856988 26198 heartbeater.cc:347] Connected to a master server at 127.17.96.254:45519
I20260619 02:30:08.857337 26198 heartbeater.cc:464] Registering TS with master...
I20260619 02:30:08.858189 26198 heartbeater.cc:511] Master 127.17.96.254:45519 requested a full tablet report, sending...
I20260619 02:30:08.860606 26007 ts_manager.cc:194] Registered new tserver with Master: 46241c037aca4af99fb2015c12c4e268 (127.17.96.193:39975)
I20260619 02:30:08.862244 26007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:60671
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
I20260619 02:30:08.870064 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45519
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:30:09.004283 26203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:09.004642 26203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:09.004724 26203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:09.009346 26203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:09.009455 26203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:09.009560 26203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:30:09.016052 26203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:09.017431 26203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:09.019213 26203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:09.031180 26209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:09.031224 26211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:09.031180 26208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:09.031553 26203 server_base.cc:1034] running on GCE node
I20260619 02:30:09.032217 26203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:09.033083 26203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:09.034365 26203 hybrid_clock.cc:648] HybridClock initialized: now 1781836209034303 us; error 81 us; skew 500 ppm
Jun 19 02:30:09 dist-test-slave-4qw6 krb5kdc[25962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836209, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:30:09.038008 26203 init.cc:377] Logged in from keytab as kudu/127.17.96.194@KRBTEST.COM (short username kudu)
I20260619 02:30:09.039266 26203 webserver.cc:460] Webserver started at http://127.17.96.194:32833/ using document root <none> and password file <none>
I20260619 02:30:09.039988 26203 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:09.040103 26203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:09.040383 26203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:30:09.042968 26203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "4892f42b143f4ef8bfbc99d4b7672a70"
format_stamp: "Formatted at 2026-06-19 02:30:09 on dist-test-slave-4qw6"
server_key: "854a9f875c1d680ea1ea53a5269bbb7e"
server_key_iv: "783384db0452a55d6d5cb6b42fafca33"
server_key_version: "encryptionkey@0"
I20260619 02:30:09.043699 26203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "4892f42b143f4ef8bfbc99d4b7672a70"
format_stamp: "Formatted at 2026-06-19 02:30:09 on dist-test-slave-4qw6"
server_key: "854a9f875c1d680ea1ea53a5269bbb7e"
server_key_iv: "783384db0452a55d6d5cb6b42fafca33"
server_key_version: "encryptionkey@0"
I20260619 02:30:09.048425 26203 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:30:09.051646 26218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:09.053284 26203 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:30:09.053421 26203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "4892f42b143f4ef8bfbc99d4b7672a70"
format_stamp: "Formatted at 2026-06-19 02:30:09 on dist-test-slave-4qw6"
server_key: "854a9f875c1d680ea1ea53a5269bbb7e"
server_key_iv: "783384db0452a55d6d5cb6b42fafca33"
server_key_version: "encryptionkey@0"
I20260619 02:30:09.053601 26203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:09.068220 26203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:09.072273 26203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:09.072530 26203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:09.073248 26203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:30:09.074340 26203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:30:09.074396 26203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:09.074481 26203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:30:09.074523 26203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:09.086020 26203 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:46017
I20260619 02:30:09.086037 26331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:46017 every 8 connection(s)
I20260619 02:30:09.087126 26203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:30:09.089458 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 26203
I20260619 02:30:09.089600 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:30:09.089994 17795 external_mini_cluster.cc:1428] Setting key af60b5ad763742248bc0798f0cb19154
Jun 19 02:30:09 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836209, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.96.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.195@KRBTEST.COM" created.
I20260619 02:30:09.104463 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:09.089312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:52249 (local address 127.17.96.254:45519)
0619 02:30:09.089469 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:09.089473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:09.090823 (+ 1350us) server_negotiation.cc:407] Connection header received
0619 02:30:09.092409 (+ 1586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:09.092413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:09.092466 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:09.092548 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:09.094211 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:09.094809 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:09.096144 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:09.096417 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:09.096548 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:09.099888 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:09.099892 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:09.099895 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:30:09.099934 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:09.101679 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:09.102407 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:09.102411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:09.102412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:09.102506 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:09.102947 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:09.102950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:09.102951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:09.103298 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:09.103419 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:09.104091 (+ 672us) server_negotiation.cc:299] Negotiation successful
0619 02:30:09.104265 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 02:30:09.105659 26334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:09.089979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.194:52249)
0619 02:30:09.090623 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:09.090707 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:09.092173 (+ 1466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:09.092761 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:09.092769 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:09.093194 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:09.094011 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:09.094033 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:09.094963 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:09.094973 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:09.095987 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:09.095999 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:09.096603 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:09.096608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:09.096810 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:09.097549 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:09.097569 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:09.099643 (+ 2074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:30:09.101872 (+ 2229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:09.101879 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:09.101896 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:09.102230 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:09.102608 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:09.102611 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:09.102613 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:09.102838 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:09.103419 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:30:09.103425 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:30:09.103734 (+ 309us) client_negotiation.cc:769] Sending connection context
0619 02:30:09.104027 (+ 293us) client_negotiation.cc:240] Negotiation successful
0619 02:30:09.104426 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":153,"threads_started":1}
I20260619 02:30:09.107158 26332 heartbeater.cc:347] Connected to a master server at 127.17.96.254:45519
I20260619 02:30:09.107472 26332 heartbeater.cc:464] Registering TS with master...
I20260619 02:30:09.108197 26332 heartbeater.cc:511] Master 127.17.96.254:45519 requested a full tablet report, sending...
I20260619 02:30:09.109603 26007 ts_manager.cc:194] Registered new tserver with Master: 4892f42b143f4ef8bfbc99d4b7672a70 (127.17.96.194:46017)
I20260619 02:30:09.110425 26007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:52249
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
I20260619 02:30:09.125452 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45519
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:30:09.264812 26337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:09.265245 26337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:09.265313 26337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:09.270264 26337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:09.270395 26337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:09.270473 26337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:30:09.278267 26337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:09.279568 26337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:09.280684 26337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:09.292893 26345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:09.292886 26342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:09.292886 26343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:09.293303 26337 server_base.cc:1034] running on GCE node
I20260619 02:30:09.293941 26337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:09.294797 26337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:09.296290 26337 hybrid_clock.cc:648] HybridClock initialized: now 1781836209296251 us; error 97 us; skew 500 ppm
Jun 19 02:30:09 dist-test-slave-4qw6 krb5kdc[25962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836209, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:30:09.300207 26337 init.cc:377] Logged in from keytab as kudu/127.17.96.195@KRBTEST.COM (short username kudu)
I20260619 02:30:09.301455 26337 webserver.cc:460] Webserver started at http://127.17.96.195:45201/ using document root <none> and password file <none>
I20260619 02:30:09.302376 26337 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:09.302464 26337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:09.302767 26337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:30:09.305397 26337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "aafacb05ceca4a03a81d80a47602bd6e"
format_stamp: "Formatted at 2026-06-19 02:30:09 on dist-test-slave-4qw6"
server_key: "22e5bcdbea3fb88b0010cc27d4661e1e"
server_key_iv: "2a7ae72bf5ed85451504e3d644279c24"
server_key_version: "encryptionkey@0"
I20260619 02:30:09.306222 26337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "aafacb05ceca4a03a81d80a47602bd6e"
format_stamp: "Formatted at 2026-06-19 02:30:09 on dist-test-slave-4qw6"
server_key: "22e5bcdbea3fb88b0010cc27d4661e1e"
server_key_iv: "2a7ae72bf5ed85451504e3d644279c24"
server_key_version: "encryptionkey@0"
I20260619 02:30:09.311299 26337 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260619 02:30:09.314608 26352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:09.316429 26337 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:30:09.316654 26337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "aafacb05ceca4a03a81d80a47602bd6e"
format_stamp: "Formatted at 2026-06-19 02:30:09 on dist-test-slave-4qw6"
server_key: "22e5bcdbea3fb88b0010cc27d4661e1e"
server_key_iv: "2a7ae72bf5ed85451504e3d644279c24"
server_key_version: "encryptionkey@0"
I20260619 02:30:09.316814 26337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:09.336032 26337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:09.340471 26337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:09.340809 26337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:09.342168 26337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:30:09.343566 26337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:30:09.343660 26337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:09.343719 26337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:30:09.343741 26337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:09.356302 26337 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:39309
I20260619 02:30:09.356427 26465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:39309 every 8 connection(s)
I20260619 02:30:09.357553 26337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:30:09.365586 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 26337
I20260619 02:30:09.365732 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:30:09.366146 17795 external_mini_cluster.cc:1428] Setting key 08cf96f1c01592a12a3ae60dfe4c3434
Jun 19 02:30:09 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836209, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:30:09.376338 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:09.360070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:37499 (local address 127.17.96.254:45519)
0619 02:30:09.360293 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:09.360298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:09.361109 (+ 811us) server_negotiation.cc:407] Connection header received
0619 02:30:09.363163 (+ 2054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:09.363168 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:09.363260 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:09.363355 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:09.365123 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:09.365929 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:09.367901 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:09.368192 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:09.368362 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:09.372037 (+ 3675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:09.372046 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:09.372049 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:30:09.372082 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:09.373739 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:09.374440 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:09.374444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:09.374446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:09.374538 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:09.374971 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:09.374977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:09.374978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:09.375226 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:09.375352 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:09.375980 (+ 628us) server_negotiation.cc:299] Negotiation successful
0619 02:30:09.376152 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 02:30:09.377406 26468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:09.360221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.195:37499)
0619 02:30:09.360871 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:09.360948 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:09.362811 (+ 1863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:09.363625 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:09.363636 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:09.364175 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:09.364910 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:09.364929 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:09.366096 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:09.366101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:09.367751 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:09.367765 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:09.368475 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:09.368481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:09.368772 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:09.369694 (+ 922us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:09.369715 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:09.371722 (+ 2007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:30:09.373950 (+ 2228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:09.373957 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:09.373979 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:09.374303 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:09.374654 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:09.374657 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:09.374659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:09.374857 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:09.375363 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:30:09.375373 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:30:09.375700 (+ 327us) client_negotiation.cc:769] Sending connection context
0619 02:30:09.375926 (+ 226us) client_negotiation.cc:240] Negotiation successful
0619 02:30:09.376333 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":142,"threads_started":1}
I20260619 02:30:09.379137 26466 heartbeater.cc:347] Connected to a master server at 127.17.96.254:45519
I20260619 02:30:09.379438 26466 heartbeater.cc:464] Registering TS with master...
I20260619 02:30:09.380044 26466 heartbeater.cc:511] Master 127.17.96.254:45519 requested a full tablet report, sending...
I20260619 02:30:09.381255 26007 ts_manager.cc:194] Registered new tserver with Master: aafacb05ceca4a03a81d80a47602bd6e (127.17.96.195:39309)
I20260619 02:30:09.382074 26007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:37499
I20260619 02:30:09.394927 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.96.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.96.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.96.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.193.keytab.
Entry for principal oryx/127.17.96.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.193.keytab.
WARNING: no policy specified for oryx/127.17.96.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.96.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.96.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.194.keytab.
Entry for principal oryx/127.17.96.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.194.keytab.
WARNING: no policy specified for oryx/127.17.96.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.96.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.96.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.195.keytab.
Entry for principal oryx/127.17.96.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.195.keytab.
I20260619 02:30:09.482877 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 26069
I20260619 02:30:09.489236 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 26203
I20260619 02:30:09.495548 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 26337
I20260619 02:30:09.502296 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 25975
I20260619 02:30:09.510351 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:45519
--webserver_interface=127.17.96.254
--webserver_port=34571
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:45519
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:30:09.640152 26475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:09.640547 26475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:09.640599 26475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:09.645296 26475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:30:09.645408 26475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:09.645435 26475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:09.645458 26475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:30:09.645485 26475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:30:09.651765 26475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:45519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:45519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=34571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:09.653040 26475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:09.654246 26475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:09.664465 26483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:09.664486 26481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:09.664723 26475 server_base.cc:1034] running on GCE node
W20260619 02:30:09.664465 26480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:09.665613 26475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:09.667054 26475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:09.668291 26475 hybrid_clock.cc:648] HybridClock initialized: now 1781836209668268 us; error 49 us; skew 500 ppm
Jun 19 02:30:09 dist-test-slave-4qw6 krb5kdc[25962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836209, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:30:09.671916 26475 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:30:09.673055 26475 webserver.cc:460] Webserver started at http://127.17.96.254:34571/ using document root <none> and password file <none>
I20260619 02:30:09.673735 26475 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:09.673928 26475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:09.678826 26475 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260619 02:30:09.681596 26490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:09.683199 26475 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 02:30:09.683409 26475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "a14cf94de8564cb69a12440f5baf5483"
format_stamp: "Formatted at 2026-06-19 02:30:08 on dist-test-slave-4qw6"
server_key: "03dff33546e504a609a2f7ed159f02a8"
server_key_iv: "673faecda15692e8a8696a6fdb5c81e9"
server_key_version: "encryptionkey@0"
I20260619 02:30:09.684087 26475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:09.706462 26475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:09.710593 26475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:09.710875 26475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:09.719519 26475 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:45519
I20260619 02:30:09.719537 26542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:45519 every 8 connection(s)
I20260619 02:30:09.720834 26475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:30:09.727233 26543 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:30:09.729660 26543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483: Bootstrap starting.
I20260619 02:30:09.730511 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 26475
I20260619 02:30:09.731168 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:39975
--local_ip_for_outbound_sockets=127.17.96.193
--tserver_master_addrs=127.17.96.254:45519
--webserver_port=38089
--webserver_interface=127.17.96.193
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.96.193
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:30:09.753736 26543 log.cc:824] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483: Log is configured to *not* fsync() on all Append() calls
I20260619 02:30:09.763872 26543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:30:09.764389 26543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483: Bootstrap complete.
I20260619 02:30:09.769464 26543 raft_consensus.cc:348] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14cf94de8564cb69a12440f5baf5483" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45519 } }
I20260619 02:30:09.770110 26543 raft_consensus.cc:729] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a14cf94de8564cb69a12440f5baf5483, State: Initialized, Role: FOLLOWER
I20260619 02:30:09.770732 26543 consensus_queue.cc:260] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14cf94de8564cb69a12440f5baf5483" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45519 } }
I20260619 02:30:09.770906 26543 raft_consensus.cc:388] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:30:09.770998 26543 raft_consensus.cc:482] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:30:09.771135 26543 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:30:09.773447 26543 raft_consensus.cc:504] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14cf94de8564cb69a12440f5baf5483" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45519 } }
I20260619 02:30:09.773929 26543 leader_election.cc:302] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a14cf94de8564cb69a12440f5baf5483; no voters:
I20260619 02:30:09.774356 26543 leader_election.cc:288] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 02:30:09.774461 26547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 02:30:09.774662 26547 raft_consensus.cc:686] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [term 2 LEADER]: Becoming Leader. State: Replica: a14cf94de8564cb69a12440f5baf5483, State: Running, Role: LEADER
I20260619 02:30:09.774994 26547 consensus_queue.cc:237] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14cf94de8564cb69a12440f5baf5483" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45519 } }
I20260619 02:30:09.775753 26543 sys_catalog.cc:564] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:30:09.776471 26548 sys_catalog.cc:455] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a14cf94de8564cb69a12440f5baf5483" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14cf94de8564cb69a12440f5baf5483" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45519 } } }
I20260619 02:30:09.776628 26548 sys_catalog.cc:458] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [sys.catalog]: This master's current role is: LEADER
I20260619 02:30:09.777415 26554 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:30:09.777510 26549 sys_catalog.cc:455] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a14cf94de8564cb69a12440f5baf5483. Latest consensus state: current_term: 2 leader_uuid: "a14cf94de8564cb69a12440f5baf5483" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a14cf94de8564cb69a12440f5baf5483" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45519 } } }
I20260619 02:30:09.777644 26549 sys_catalog.cc:458] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483 [sys.catalog]: This master's current role is: LEADER
I20260619 02:30:09.781467 26554 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:30:09.782528 26554 catalog_manager.cc:1259] Loaded cluster ID: 5a85a589e0e9400791c32ce673dc5ec2
I20260619 02:30:09.782621 26554 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:30:09.785617 26554 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:30:09.786511 26554 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a14cf94de8564cb69a12440f5baf5483: Loaded TSK: 0
I20260619 02:30:09.787945 26554 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:30:09.866920 26545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:09.867252 26545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:09.867304 26545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:09.872351 26545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:09.872452 26545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:09.872521 26545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:30:09.879240 26545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.193.keytab
--principal=oryx/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:39975
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=38089
--tserver_master_addrs=127.17.96.254:45519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:09.880517 26545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:09.881796 26545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:09.893381 26569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:09.893602 26572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:09.893410 26570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:09.893816 26545 server_base.cc:1034] running on GCE node
I20260619 02:30:09.894322 26545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:09.895167 26545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:09.896373 26545 hybrid_clock.cc:648] HybridClock initialized: now 1781836209896355 us; error 44 us; skew 500 ppm
Jun 19 02:30:09 dist-test-slave-4qw6 krb5kdc[25962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836209, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:30:09.899781 26545 init.cc:377] Logged in from keytab as oryx/127.17.96.193@KRBTEST.COM (short username oryx)
I20260619 02:30:09.900978 26545 webserver.cc:460] Webserver started at http://127.17.96.193:38089/ using document root <none> and password file <none>
I20260619 02:30:09.901634 26545 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:09.901731 26545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:09.906598 26545 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:30:09.909301 26579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:09.910984 26545 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:30:09.911118 26545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "46241c037aca4af99fb2015c12c4e268"
format_stamp: "Formatted at 2026-06-19 02:30:08 on dist-test-slave-4qw6"
server_key: "f0b6c1b8c1e7134a5bd6601cbf69bb91"
server_key_iv: "3dd1f825901a7472f91c7841cab1c381"
server_key_version: "encryptionkey@0"
I20260619 02:30:09.911839 26545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:09.925433 26545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:09.930447 26545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:09.930706 26545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:09.931453 26545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:30:09.932480 26545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:30:09.932536 26545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:09.932621 26545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:30:09.932655 26545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:09.942574 26545 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:39975
I20260619 02:30:09.942677 26692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:39975 every 8 connection(s)
I20260619 02:30:09.943835 26545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:30:09.950857 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 26545
I20260619 02:30:09.951464 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:46017
--local_ip_for_outbound_sockets=127.17.96.194
--tserver_master_addrs=127.17.96.254:45519
--webserver_port=32833
--webserver_interface=127.17.96.194
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.96.194
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:30:09 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:09 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
W20260619 02:30:09.961311 26695 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:09.946342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:55949 (local address 127.17.96.254:45519)
0619 02:30:09.946921 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:09.946943 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:09.947351 (+ 408us) server_negotiation.cc:407] Connection header received
0619 02:30:09.949107 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:09.949119 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:09.949458 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:09.949854 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:09.951453 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:09.952679 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:09.954184 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:09.954749 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:09.954905 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:09.960415 (+ 5510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.193:55949: BlockingRecv error: recv got EOF from 127.17.96.193:55949 (error 108)
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":149,"threads_started":1}
I20260619 02:30:09.961478 26696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:09.946574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.193:55949)
0619 02:30:09.947161 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:09.947210 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:09.948602 (+ 1392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:09.949941 (+ 1339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:09.949950 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:09.950455 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:09.951223 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:09.951252 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:09.952894 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:09.952897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:09.953995 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:09.954029 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:09.954974 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:09.954981 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:09.955268 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:09.956119 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:09.956153 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:09.960125 (+ 3972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":158,"threads_started":1}
W20260619 02:30:09.962175 26693 heartbeater.cc:644] Failed to heartbeat to 127.17.96.254:45519 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.96.254:45519: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Jun 19 02:30:09 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:09 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:09.969715 26696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:09.962832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.193:46673)
0619 02:30:09.963068 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:09.963090 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:09.963202 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:09.963809 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:09.963812 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:09.964033 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:09.964226 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:09.964239 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:09.965134 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:09.965141 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:09.965844 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:09.965860 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:09.966394 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:09.966399 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:09.966519 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:09.967179 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:09.967194 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:09.969541 (+ 2347us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150}
W20260619 02:30:09.969977 26695 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:09.963404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:46673 (local address 127.17.96.254:45519)
0619 02:30:09.963553 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:09.963557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:09.963572 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:30:09.963619 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:09.963621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:09.963669 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:09.963791 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:09.964387 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:09.964988 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:09.965998 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:09.966250 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:09.966391 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:09.969795 (+ 3404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.193:46673: BlockingRecv error: recv got EOF from 127.17.96.193:46673 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Jun 19 02:30:09 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:09 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:09.977653 26696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:09.970600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.193:46443)
0619 02:30:09.970755 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:09.970772 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:09.970869 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:09.971160 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:09.971164 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:09.971375 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:09.971489 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:09.971496 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:09.972297 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:09.972301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:09.973308 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:09.973322 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:09.973929 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:09.973934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:09.974039 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:09.974638 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:09.974659 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:09.977493 (+ 2834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":62}
W20260619 02:30:09.978341 26693 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:30:09.978509 26695 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:09.970734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:46443 (local address 127.17.96.254:45519)
0619 02:30:09.970867 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:09.970870 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:09.970887 (+ 17us) server_negotiation.cc:407] Connection header received
0619 02:30:09.970977 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:09.970979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:09.971029 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:09.971115 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:09.971616 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:09.972167 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:09.973466 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:09.973708 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:09.973903 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:09.978232 (+ 4329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.193:46443: BlockingRecv error: recv got EOF from 127.17.96.193:46443 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260619 02:30:10.084853 26697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:10.085208 26697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:10.085258 26697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:10.090312 26697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:10.090405 26697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:10.090479 26697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:30:10.096937 26697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.194.keytab
--principal=oryx/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:46017
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=32833
--tserver_master_addrs=127.17.96.254:45519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:10.098357 26697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:10.099604 26697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:10.111410 26703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:10.111606 26697 server_base.cc:1034] running on GCE node
W20260619 02:30:10.111480 26702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:10.111480 26705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:10.112514 26697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:10.113557 26697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:10.114779 26697 hybrid_clock.cc:648] HybridClock initialized: now 1781836210114753 us; error 50 us; skew 500 ppm
Jun 19 02:30:10 dist-test-slave-4qw6 krb5kdc[25962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836210, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:30:10.118608 26697 init.cc:377] Logged in from keytab as oryx/127.17.96.194@KRBTEST.COM (short username oryx)
I20260619 02:30:10.119827 26697 webserver.cc:460] Webserver started at http://127.17.96.194:32833/ using document root <none> and password file <none>
I20260619 02:30:10.120502 26697 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:10.120591 26697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:10.125844 26697 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 02:30:10.128751 26712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:10.130664 26697 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 02:30:10.130844 26697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "4892f42b143f4ef8bfbc99d4b7672a70"
format_stamp: "Formatted at 2026-06-19 02:30:09 on dist-test-slave-4qw6"
server_key: "854a9f875c1d680ea1ea53a5269bbb7e"
server_key_iv: "783384db0452a55d6d5cb6b42fafca33"
server_key_version: "encryptionkey@0"
I20260619 02:30:10.131500 26697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:10.151350 26697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:10.155424 26697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:10.155632 26697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:10.156308 26697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:30:10.157287 26697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:30:10.157346 26697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:10.157395 26697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:30:10.157413 26697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:10.168361 26697 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:46017
I20260619 02:30:10.168386 26825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:46017 every 8 connection(s)
I20260619 02:30:10.169653 26697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:30:10.170172 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 26697
I20260619 02:30:10.171124 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:39309
--local_ip_for_outbound_sockets=127.17.96.195
--tserver_master_addrs=127.17.96.254:45519
--webserver_port=45201
--webserver_interface=127.17.96.195
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:45631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.96.195
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:30:10 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:10 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
W20260619 02:30:10.188824 26695 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:10.172861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:59071 (local address 127.17.96.254:45519)
0619 02:30:10.173046 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:10.173051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:10.174468 (+ 1417us) server_negotiation.cc:407] Connection header received
0619 02:30:10.176961 (+ 2493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:10.176969 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:10.177093 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:10.177270 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:10.179519 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.180110 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.182514 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.182837 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.183126 (+ 289us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:10.188623 (+ 5497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.194:59071: BlockingRecv error: recv got EOF from 127.17.96.194:59071 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260619 02:30:10.189935 26829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:10.173357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.194:59071)
0619 02:30:10.174159 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:10.174263 (+ 104us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:10.176558 (+ 2295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:10.177536 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:10.177553 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:10.178315 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:10.179278 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:10.179308 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.180295 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.180302 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:10.182233 (+ 1931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:10.182253 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.183200 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.183209 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:10.183517 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:10.184490 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:10.184521 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:10.188407 (+ 3886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":141,"threads_started":1}
W20260619 02:30:10.190725 26826 heartbeater.cc:644] Failed to heartbeat to 127.17.96.254:45519 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.96.254:45519: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Jun 19 02:30:10 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:10 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:10.198153 26829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:10.191457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.194:56755)
0619 02:30:10.191631 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:10.191650 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:10.191808 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:10.192111 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:10.192114 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:10.192326 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:10.192448 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:10.192454 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.193302 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.193308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:10.193962 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:10.193970 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.194454 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.194458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:10.194565 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:10.195344 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:10.195364 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:10.197976 (+ 2612us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":62}
W20260619 02:30:10.198419 26695 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:10.191555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:56755 (local address 127.17.96.254:45519)
0619 02:30:10.191711 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:10.191718 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:10.191738 (+ 20us) server_negotiation.cc:407] Connection header received
0619 02:30:10.191935 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:10.191938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:10.191997 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:10.192092 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:10.192576 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.193151 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.194083 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.194327 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.194447 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:10.198247 (+ 3800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.194:56755: BlockingRecv error: recv got EOF from 127.17.96.194:56755 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Jun 19 02:30:10 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:10 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:10.204988 26829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:10.198849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.194:52927)
0619 02:30:10.199043 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:10.199060 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:10.199163 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:10.199466 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:10.199469 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:10.199672 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:10.199800 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:10.199807 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.200640 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.200643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:10.201514 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:10.201527 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.202064 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.202068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:10.202157 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:10.202701 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:10.202716 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:10.204855 (+ 2139us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115}
W20260619 02:30:10.205345 26826 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:30:10.205354 26695 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:10.198954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:52927 (local address 127.17.96.254:45519)
0619 02:30:10.199098 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:10.199101 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:10.199116 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:30:10.199273 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:10.199275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:10.199327 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:10.199408 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:10.199923 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.200502 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.201658 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.201947 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.202036 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:10.205069 (+ 3033us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.194:52927: BlockingRecv error: recv got EOF from 127.17.96.194:52927 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260619 02:30:10.309473 26828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:10.309895 26828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:10.309962 26828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:10.314816 26828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:10.314906 26828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:10.314985 26828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:30:10.321391 26828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:45631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/krb5kdc/oryx_127.17.96.195.keytab
--principal=oryx/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:39309
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=45201
--tserver_master_addrs=127.17.96.254:45519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:10.322729 26828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:10.323915 26828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:10.336062 26834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:10.336062 26837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:10.336218 26835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:10.336355 26828 server_base.cc:1034] running on GCE node
I20260619 02:30:10.337069 26828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:10.337999 26828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:10.339213 26828 hybrid_clock.cc:648] HybridClock initialized: now 1781836210339200 us; error 42 us; skew 500 ppm
Jun 19 02:30:10 dist-test-slave-4qw6 krb5kdc[25962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836210, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:30:10.342888 26828 init.cc:377] Logged in from keytab as oryx/127.17.96.195@KRBTEST.COM (short username oryx)
I20260619 02:30:10.344152 26828 webserver.cc:460] Webserver started at http://127.17.96.195:45201/ using document root <none> and password file <none>
I20260619 02:30:10.345176 26828 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:10.345322 26828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:10.350667 26828 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 02:30:10.353629 26844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:10.355511 26828 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 02:30:10.355674 26828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "aafacb05ceca4a03a81d80a47602bd6e"
format_stamp: "Formatted at 2026-06-19 02:30:09 on dist-test-slave-4qw6"
server_key: "22e5bcdbea3fb88b0010cc27d4661e1e"
server_key_iv: "2a7ae72bf5ed85451504e3d644279c24"
server_key_version: "encryptionkey@0"
I20260619 02:30:10.356405 26828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:10.376879 26828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:10.380625 26828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:10.380877 26828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:10.381582 26828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:30:10.382632 26828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:30:10.382716 26828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:10.382815 26828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:30:10.382863 26828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:10.394629 26828 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:39309
I20260619 02:30:10.394688 26957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:39309 every 8 connection(s)
I20260619 02:30:10.395946 26828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:30:10.402182 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 26828
Jun 19 02:30:10 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:10 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
W20260619 02:30:10.409668 26695 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:10.398407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:43221 (local address 127.17.96.254:45519)
0619 02:30:10.398583 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:10.398587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:10.399910 (+ 1323us) server_negotiation.cc:407] Connection header received
0619 02:30:10.400867 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:10.400870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:10.400928 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:10.401027 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:10.402610 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.403447 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.404828 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.405091 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.405230 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:10.409508 (+ 4278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.195:43221: BlockingRecv error: recv got EOF from 127.17.96.195:43221 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 02:30:10.410528 26960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:10.398872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.195:43221)
0619 02:30:10.399705 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:10.399755 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:10.400646 (+ 891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:10.401211 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:10.401218 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:10.401618 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:10.402402 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:10.402425 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.403616 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.403621 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:10.404657 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:10.404669 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.405281 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.405286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:10.405499 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:10.406335 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:10.406356 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:10.409366 (+ 3010us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":151,"threads_started":1}
W20260619 02:30:10.411090 26958 heartbeater.cc:644] Failed to heartbeat to 127.17.96.254:45519 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.96.254:45519: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
I20260619 02:30:10.413709 26962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:10.403499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58630 (local address 127.17.96.254:45519)
0619 02:30:10.403782 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:10.403786 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:10.403798 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:30:10.403831 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:10.403834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:10.403874 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:10.403998 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:10.404807 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.405376 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.406390 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.406640 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.406773 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:10.408358 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:10.408365 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:10.408383 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:30:10.408413 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:10.410665 (+ 2252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:10.411827 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:10.411832 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:10.411838 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:10.411978 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:10.412314 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:10.412323 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:10.412324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:10.412721 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:10.412965 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:10.413290 (+ 325us) server_negotiation.cc:299] Negotiation successful
0619 02:30:10.413572 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":130,"threads_started":1}
Jun 19 02:30:10 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:10 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:10.418768 26960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:10.411638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.195:47335)
0619 02:30:10.411769 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:10.411789 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:10.411895 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:10.412222 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:10.412225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:10.412412 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:10.412532 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:10.412538 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.413385 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.413392 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:10.414352 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:10.414366 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.414853 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.414860 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:10.414993 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:10.415943 (+ 950us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:10.415969 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:10.418657 (+ 2688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 02:30:10.418869 26695 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:10.411732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:47335 (local address 127.17.96.254:45519)
0619 02:30:10.411867 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:10.411872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:10.411888 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:30:10.412038 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:10.412042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:10.412101 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:10.412204 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:10.412666 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.413240 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.414493 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.414722 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.414815 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:10.418775 (+ 3960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.195:47335: BlockingRecv error: recv got EOF from 127.17.96.195:47335 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 19 02:30:10 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:10 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:10.425537 26960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:10.419591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.195:55267)
0619 02:30:10.419713 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:10.419732 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:10.419823 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:10.420055 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:10.420057 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:10.420209 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:10.420299 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:10.420305 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.421116 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.421119 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:10.421716 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:10.421726 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.422403 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.422407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:10.422477 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:10.422952 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:10.422965 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:10.425376 (+ 2411us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260619 02:30:10.425923 26695 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:10.419645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:55267 (local address 127.17.96.254:45519)
0619 02:30:10.419790 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:10.419797 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:10.419816 (+ 19us) server_negotiation.cc:407] Connection header received
0619 02:30:10.419907 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:10.419909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:10.419948 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:10.420021 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:10.420399 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.420920 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.421927 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.422262 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.422351 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:10.425558 (+ 3207us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.195:55267: BlockingRecv error: recv got EOF from 127.17.96.195:55267 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260619 02:30:10.425923 26958 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 02:30:10 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:10 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:10.986523 26963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:10.979087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.193:54015)
0619 02:30:10.979371 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:10.979386 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:10.979491 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:10.979746 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:10.979748 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:10.979959 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:10.980061 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:10.980067 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.980882 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.980884 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:10.981509 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:10.981519 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.982123 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.982125 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:10.982194 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:10.982851 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:10.982866 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:10.986314 (+ 3448us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":97,"threads_started":1}
W20260619 02:30:10.986683 26964 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:10.979128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:54015 (local address 127.17.96.254:45519)
0619 02:30:10.979460 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:10.979464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:10.979479 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:30:10.979599 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:10.979601 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:10.979648 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:10.979738 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:10.980178 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.980763 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.981646 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:10.981997 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:10.982101 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:10.986524 (+ 4423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.193:54015: BlockingRecv error: recv got EOF from 127.17.96.193:54015 (error 108)
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":126,"threads_started":1}
Jun 19 02:30:11 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:11 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:11.213848 26965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:11.206515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.194:59005)
0619 02:30:11.206929 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:11.206950 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:11.207113 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:11.207422 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:11.207425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:11.207633 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:11.207741 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:11.207748 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:11.208643 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:11.208646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:11.209271 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:11.209280 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:11.209850 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:11.209854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:11.209923 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:11.210512 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:11.210525 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:11.213600 (+ 3075us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":122,"threads_started":1}
W20260619 02:30:11.214090 26964 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:11.206716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:59005 (local address 127.17.96.254:45519)
0619 02:30:11.206955 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:11.206960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:11.207109 (+ 149us) server_negotiation.cc:407] Connection header received
0619 02:30:11.207209 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:11.207212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:11.207275 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:11.207376 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:11.207868 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:11.208507 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:11.209408 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:11.209648 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:11.209825 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:11.213924 (+ 4099us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.194:59005: BlockingRecv error: recv got EOF from 127.17.96.194:59005 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Jun 19 02:30:11 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:11 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:11.434168 26966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:11.426775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.195:44405)
0619 02:30:11.427138 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:11.427158 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:11.427317 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:11.427693 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:11.427698 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:11.427977 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:11.428165 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:11.428174 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:11.429023 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:11.429026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:11.429678 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:11.429687 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:11.430279 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:11.430282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:11.430355 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:11.431061 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:11.431076 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:11.434003 (+ 2927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":97,"threads_started":1}
W20260619 02:30:11.434355 26964 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:11.426915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:44405 (local address 127.17.96.254:45519)
0619 02:30:11.427084 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:11.427091 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:11.427264 (+ 173us) server_negotiation.cc:407] Connection header received
0619 02:30:11.427462 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:11.427469 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:11.427550 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:11.427627 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:11.428298 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:11.428901 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:11.429907 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:11.430156 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:11.430249 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:11.434195 (+ 3946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.195:44405: BlockingRecv error: recv got EOF from 127.17.96.195:44405 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Jun 19 02:30:11 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:11 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:11.995133 26968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:11.988016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.193:47843)
0619 02:30:11.988391 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:11.988410 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:11.988589 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:11.988938 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:11.988941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:11.989192 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:11.989348 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:11.989357 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:11.990295 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:11.990301 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:11.990946 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:11.990956 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:11.991684 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:11.991689 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:11.991792 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:11.992471 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:11.992490 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:11.994960 (+ 2470us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":114,"threads_started":1}
W20260619 02:30:11.995579 26967 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:11.988032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:47843 (local address 127.17.96.254:45519)
0619 02:30:11.988391 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:11.988398 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:11.988515 (+ 117us) server_negotiation.cc:407] Connection header received
0619 02:30:11.988744 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:11.988746 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:11.988797 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:11.988910 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:11.989499 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:11.990162 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:11.991142 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:11.991489 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:11.991646 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:11.995389 (+ 3743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.193:47843: BlockingRecv error: recv got EOF from 127.17.96.193:47843 (error 108)
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":124,"threads_started":1}
Jun 19 02:30:12 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:12 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:12.222784 26969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:12.215097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.194:37283)
0619 02:30:12.215485 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:12.215513 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:12.215685 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:12.216100 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:12.216104 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:12.216403 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:12.216600 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:12.216610 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:12.217779 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:12.217786 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:12.218613 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:12.218627 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:12.219233 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:12.219236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:12.219324 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:12.220009 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:12.220022 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:12.222598 (+ 2576us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":153,"threads_started":1}
W20260619 02:30:12.223006 26967 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:12.215097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:37283 (local address 127.17.96.254:45519)
0619 02:30:12.215487 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:12.215491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:12.215645 (+ 154us) server_negotiation.cc:407] Connection header received
0619 02:30:12.215797 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:12.215800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:12.215889 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:12.215996 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:12.216761 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:12.217590 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:12.218845 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:12.219093 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:12.219224 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:12.222802 (+ 3578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.194:37283: BlockingRecv error: recv got EOF from 127.17.96.194:37283 (error 108)
Metrics: {"server-negotiator.queue_time_us":228}
Jun 19 02:30:12 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:12 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:12.442930 26970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:12.435543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.195:59663)
0619 02:30:12.435973 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:12.435991 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:12.436170 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:12.436613 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:12.436616 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:12.436895 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:12.437038 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:12.437045 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:12.438124 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:12.438127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:12.438785 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:12.438797 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:12.439437 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:12.439446 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:12.439561 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:12.440326 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:12.440349 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:12.442651 (+ 2302us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":178,"threads_started":1}
W20260619 02:30:12.443097 26967 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:12.435781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:59663 (local address 127.17.96.254:45519)
0619 02:30:12.435949 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:12.435953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:12.436081 (+ 128us) server_negotiation.cc:407] Connection header received
0619 02:30:12.436293 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:12.436296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:12.436353 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:12.436484 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:12.437228 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:12.437944 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:12.438937 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:12.439191 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:12.439294 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:12.442913 (+ 3619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.195:59663: BlockingRecv error: recv got EOF from 127.17.96.195:59663 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 19 02:30:12 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:12 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:13.003789 26971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:12.996402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.193:36911)
0619 02:30:12.996821 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:12.996838 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:12.996967 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:12.997607 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:12.997612 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:12.997910 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:12.998092 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:12.998103 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:12.999219 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:12.999226 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:13.000036 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:13.000053 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:13.000557 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:13.000560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:13.000633 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:13.001382 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:13.001398 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:13.003586 (+ 2188us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":167,"threads_started":1}
W20260619 02:30:13.003897 26972 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:12.996548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:36911 (local address 127.17.96.254:45519)
0619 02:30:12.996892 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:12.996897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:12.996919 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:30:12.997166 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:12.997170 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:12.997232 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:12.997367 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:12.998286 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:12.999003 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:13.000172 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:13.000408 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:13.000530 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:13.003748 (+ 3218us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.193:36911: BlockingRecv error: recv got EOF from 127.17.96.193:36911 (error 108)
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":137,"threads_started":1}
Jun 19 02:30:13 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:13 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:13.231176 26973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:13.223769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.194:39111)
0619 02:30:13.224232 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:13.224250 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:13.224366 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:13.224842 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:13.224845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:13.225054 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:13.225181 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:13.225188 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:13.226264 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:13.226267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:13.227174 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:13.227189 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:13.227766 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:13.227771 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:13.227878 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:13.228608 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:13.228623 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:13.231036 (+ 2413us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":126,"threads_started":1}
W20260619 02:30:13.231426 26972 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:13.224062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:39111 (local address 127.17.96.254:45519)
0619 02:30:13.224317 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:13.224322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:13.224346 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:30:13.224533 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:13.224557 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:13.224629 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:13.224763 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:13.225387 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:13.226133 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:13.227324 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:13.227550 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:13.227644 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:13.231263 (+ 3619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.194:39111: BlockingRecv error: recv got EOF from 127.17.96.194:39111 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Jun 19 02:30:13 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:13 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:13.456166 26974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:13.443952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.195:37129)
0619 02:30:13.444445 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:13.444463 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:13.444659 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:13.444983 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:13.444986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:13.445267 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:13.445389 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:13.445395 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:13.446364 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:13.446370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:13.447216 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:13.447241 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:13.447939 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:13.447943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:13.448048 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:13.452891 (+ 4843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:13.452911 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:13.456026 (+ 3115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":126,"threads_started":1}
W20260619 02:30:13.456580 26972 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:13.444129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:37129 (local address 127.17.96.254:45519)
0619 02:30:13.444373 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:13.444379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:13.444565 (+ 186us) server_negotiation.cc:407] Connection header received
0619 02:30:13.444778 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:13.444782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:13.444854 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:13.444956 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:13.445518 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:13.446242 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:13.447491 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:13.447775 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:13.447932 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:13.456422 (+ 8490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.195:37129: BlockingRecv error: recv got EOF from 127.17.96.195:37129 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Jun 19 02:30:14 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:14 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:14.012379 26975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:14.004997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.193:37901)
0619 02:30:14.005363 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:14.005380 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:14.005498 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:14.005854 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:14.005856 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:14.006131 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:14.006249 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:14.006256 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:14.007200 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:14.007203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:14.007845 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:14.007854 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:14.008349 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:14.008351 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:14.008419 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:14.009224 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:14.009296 (+ 72us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:14.012168 (+ 2872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":137,"threads_started":1}
W20260619 02:30:14.012465 26976 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:14.005114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:37901 (local address 127.17.96.254:45519)
0619 02:30:14.005460 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:14.005464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:14.005480 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:30:14.005611 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:14.005614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:14.005668 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:14.005844 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:14.006401 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:14.007022 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:14.007987 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:14.008227 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:14.008322 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:14.012316 (+ 3994us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.193:37901: BlockingRecv error: recv got EOF from 127.17.96.193:37901 (error 108)
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":137,"threads_started":1}
Jun 19 02:30:14 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:14 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:14.238929 26977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:14.232212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.194:36537)
0619 02:30:14.232614 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:14.232635 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:14.232815 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:14.233138 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:14.233141 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:14.233394 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:14.233491 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:14.233497 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:14.234482 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:14.234489 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:14.235281 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:14.235291 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:14.235835 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:14.235839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:14.235924 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:14.236499 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:14.236512 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:14.238725 (+ 2213us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":135,"threads_started":1}
W20260619 02:30:14.239032 26976 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:14.232421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:36537 (local address 127.17.96.254:45519)
0619 02:30:14.232616 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:14.232623 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:14.232737 (+ 114us) server_negotiation.cc:407] Connection header received
0619 02:30:14.232941 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:14.232944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:14.233002 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:14.233112 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:14.233633 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:14.234363 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:14.235410 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:14.235705 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:14.235824 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:14.238898 (+ 3074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.194:36537: BlockingRecv error: recv got EOF from 127.17.96.194:36537 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Jun 19 02:30:14 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:14 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:14.464527 26978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:14.457274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.195:59567)
0619 02:30:14.457623 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:14.457638 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:14.457855 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:14.458253 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:14.458257 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:14.458513 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:14.458660 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:14.458668 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:14.459691 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:14.459694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:14.460387 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:14.460397 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:14.460934 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:14.460937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:14.461007 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:14.461908 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:14.461922 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:14.464381 (+ 2459us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":91,"threads_started":1}
W20260619 02:30:14.464847 26976 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:14.457387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:59567 (local address 127.17.96.254:45519)
0619 02:30:14.457603 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:14.457608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:14.457732 (+ 124us) server_negotiation.cc:407] Connection header received
0619 02:30:14.458008 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:14.458011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:14.458088 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:14.458191 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:14.458825 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:14.459547 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:14.460522 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:14.460820 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:14.460912 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:14.464711 (+ 3799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.195:59567: BlockingRecv error: recv got EOF from 127.17.96.195:59567 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Jun 19 02:30:15 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:15 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:15.020061 26979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:15.013609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.193:46959)
0619 02:30:15.014127 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:15.014147 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:15.014270 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:15.014561 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:15.014563 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:15.014765 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:15.014877 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:15.014882 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:15.015663 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:15.015666 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:15.016329 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:15.016339 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:15.016962 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:15.016964 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:15.017040 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:15.017811 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:15.017825 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:15.019920 (+ 2095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":163,"threads_started":1}
W20260619 02:30:15.020334 26980 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:15.013670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:46959 (local address 127.17.96.254:45519)
0619 02:30:15.014087 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:15.014092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:15.014273 (+ 181us) server_negotiation.cc:407] Connection header received
0619 02:30:15.014353 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:15.014356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:15.014435 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:15.014525 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:15.014985 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:15.015560 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:15.016502 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:15.016797 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:15.016925 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:15.020142 (+ 3217us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.193:46959: BlockingRecv error: recv got EOF from 127.17.96.193:46959 (error 108)
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":119,"threads_started":1}
Jun 19 02:30:15 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:15 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:15.247058 26981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:15.240078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.194:60949)
0619 02:30:15.240521 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:15.240542 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:15.240688 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:15.241055 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:15.241059 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:15.241276 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:15.241392 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:15.241399 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:15.242345 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:15.242348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:15.243138 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:15.243149 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:15.243765 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:15.243769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:15.243857 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:15.244656 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:15.244669 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:15.246930 (+ 2261us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":97,"threads_started":1}
W20260619 02:30:15.247251 26980 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:15.240501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:60949 (local address 127.17.96.254:45519)
0619 02:30:15.240715 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:15.240719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:15.240763 (+ 44us) server_negotiation.cc:407] Connection header received
0619 02:30:15.240815 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:15.240818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:15.240868 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:15.240951 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:15.241536 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:15.242216 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:15.243259 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:15.243613 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:15.243761 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:15.247099 (+ 3338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.194:60949: BlockingRecv error: recv got EOF from 127.17.96.194:60949 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Jun 19 02:30:15 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:15 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:15.473443 26982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:15.465665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.195:35875)
0619 02:30:15.466171 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:15.466196 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:15.466397 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:15.466735 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:15.466738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:15.467016 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:15.467130 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:15.467136 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:15.468316 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:15.468323 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:15.469037 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:15.469052 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:15.469850 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:15.469855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:15.469950 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:15.470720 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:15.470735 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:15.473232 (+ 2497us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":145,"threads_started":1}
W20260619 02:30:15.473719 26980 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:15.466045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:35875 (local address 127.17.96.254:45519)
0619 02:30:15.466245 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:15.466253 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:15.466298 (+ 45us) server_negotiation.cc:407] Connection header received
0619 02:30:15.466531 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:15.466535 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:15.466602 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:15.466704 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:15.467261 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:15.468076 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:15.469265 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:15.469654 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:15.469840 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:15.473414 (+ 3574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.195:35875: BlockingRecv error: recv got EOF from 127.17.96.195:35875 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Jun 19 02:30:16 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:16 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:16.027604 26983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:16.021061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.193:45359)
0619 02:30:16.021439 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:16.021455 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:16.021570 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:16.021904 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:16.021906 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:16.022220 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:16.022360 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:16.022367 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:16.023329 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:16.023335 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:16.023964 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:16.023974 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:16.024492 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:16.024496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:16.024588 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:16.025212 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:16.025224 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:16.027454 (+ 2230us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":117,"threads_started":1}
W20260619 02:30:16.027859 26984 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:16.021261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:45359 (local address 127.17.96.254:45519)
0619 02:30:16.021536 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:16.021540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:16.021559 (+ 19us) server_negotiation.cc:407] Connection header received
0619 02:30:16.021693 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:16.021695 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:16.021771 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:16.021870 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:16.022515 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:16.023205 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:16.024104 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:16.024362 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:16.024489 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:16.027677 (+ 3188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.193:45359: BlockingRecv error: recv got EOF from 127.17.96.193:45359 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":72,"threads_started":1}
Jun 19 02:30:16 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:16 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:16.254071 26985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:16.248275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.194:46109)
0619 02:30:16.248539 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:16.248564 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:16.248717 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:16.249029 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:16.249032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:16.249246 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:16.249344 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:16.249349 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:16.250172 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:16.250175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:16.250749 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:16.250756 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:16.251174 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:16.251177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:16.251238 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:16.251776 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:16.251789 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:16.253828 (+ 2039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":91,"threads_started":1}
W20260619 02:30:16.254451 26984 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:16.248310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:46109 (local address 127.17.96.254:45519)
0619 02:30:16.248475 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:16.248480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:16.248681 (+ 201us) server_negotiation.cc:407] Connection header received
0619 02:30:16.248837 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:16.248840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:16.248900 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:16.248989 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:16.249452 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:16.250069 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:16.250855 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:16.251073 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:16.251150 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:16.254021 (+ 2871us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.194:46109: BlockingRecv error: recv got EOF from 127.17.96.194:46109 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 19 02:30:16 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:16 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:16.480934 26986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:16.474393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.195:52663)
0619 02:30:16.474653 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:16.474667 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:16.474777 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:16.475086 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:16.475089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:16.475291 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:16.475397 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:16.475403 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:16.476191 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:16.476194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:16.476887 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:16.476898 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:16.477487 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:16.477491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:16.477582 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:16.478300 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:16.478320 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:16.480752 (+ 2432us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":96,"threads_started":1}
W20260619 02:30:16.481046 26984 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:16.474474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:52663 (local address 127.17.96.254:45519)
0619 02:30:16.474616 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:16.474619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:16.474716 (+ 97us) server_negotiation.cc:407] Connection header received
0619 02:30:16.474860 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:16.474862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:16.474908 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:16.474975 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:16.475507 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:16.476089 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:16.477049 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:16.477356 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:16.477481 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:16.480920 (+ 3439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.195:52663: BlockingRecv error: recv got EOF from 127.17.96.195:52663 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 19 02:30:17 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:17 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:17.034943 26987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:17.028517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.193:50081)
0619 02:30:17.028869 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:17.028884 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:17.028993 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:17.029266 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:17.029268 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:17.029471 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:17.029579 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:17.029584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:17.030438 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:17.030441 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:17.031046 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:17.031056 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:17.031580 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:17.031583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:17.031659 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:17.032221 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:17.032234 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:17.034720 (+ 2486us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":126,"threads_started":1}
W20260619 02:30:17.035105 26988 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:17.028629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:50081 (local address 127.17.96.254:45519)
0619 02:30:17.028885 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:17.028888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:17.028960 (+ 72us) server_negotiation.cc:407] Connection header received
0619 02:30:17.029079 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:17.029081 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:17.029130 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:17.029227 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:17.029700 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:17.030327 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:17.031199 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:17.031458 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:17.031557 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:17.034943 (+ 3386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.193:50081: BlockingRecv error: recv got EOF from 127.17.96.193:50081 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":92,"threads_started":1}
Jun 19 02:30:17 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:17 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:17.262849 26989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:17.255169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.194:58471)
0619 02:30:17.255523 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:17.255539 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:17.255634 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:17.255919 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:17.255921 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:17.256109 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:17.256212 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:17.256218 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:17.257010 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:17.257013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:17.257618 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:17.257627 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:17.258327 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:17.258335 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:17.258489 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:17.259358 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:17.259376 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:17.262679 (+ 3303us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":79,"threads_started":1}
W20260619 02:30:17.263422 26988 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:17.255329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:58471 (local address 127.17.96.254:45519)
0619 02:30:17.255557 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:17.255560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:17.255604 (+ 44us) server_negotiation.cc:407] Connection header received
0619 02:30:17.255737 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:17.255740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:17.255788 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:17.255878 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:17.256322 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:17.256879 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:17.257802 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:17.258037 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:17.258135 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:17.263250 (+ 5115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.194:58471: BlockingRecv error: recv got EOF from 127.17.96.194:58471 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Jun 19 02:30:17 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:17 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:17.488874 26990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:17.482238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.195:34571)
0619 02:30:17.482711 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:17.482731 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:17.482898 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:17.483289 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:17.483291 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:17.483505 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:17.483608 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:17.483613 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:17.484424 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:17.484427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:17.485015 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:17.485023 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:17.485523 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:17.485526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:17.485590 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:17.486298 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:17.486328 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:17.488744 (+ 2416us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":139,"threads_started":1}
W20260619 02:30:17.489053 26988 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:17.482360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:34571 (local address 127.17.96.254:45519)
0619 02:30:17.482587 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:17.482601 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:17.482904 (+ 303us) server_negotiation.cc:407] Connection header received
0619 02:30:17.483032 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:17.483034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:17.483110 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:17.483245 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:17.483726 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:17.484306 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:17.485165 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:17.485407 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:17.485495 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:17.488919 (+ 3424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.195:34571: BlockingRecv error: recv got EOF from 127.17.96.195:34571 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Jun 19 02:30:18 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:18 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:18.043082 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:18.036277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.193:45625)
0619 02:30:18.036624 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:18.036640 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:18.036748 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:18.037097 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:18.037099 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:18.037309 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:18.037421 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:18.037427 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:18.038372 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:18.038376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:18.039015 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:18.039025 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:18.039641 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:18.039644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:18.039769 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:18.040500 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:18.040516 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:18.042946 (+ 2430us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":138,"threads_started":1}
W20260619 02:30:18.043272 26992 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:18.036433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:45625 (local address 127.17.96.254:45519)
0619 02:30:18.036712 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:18.036716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:18.036733 (+ 17us) server_negotiation.cc:407] Connection header received
0619 02:30:18.036873 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:18.036875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:18.036963 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:18.037076 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:18.037549 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:18.038224 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:18.039166 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:18.039504 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:18.039638 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:18.043112 (+ 3474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.193:45625: BlockingRecv error: recv got EOF from 127.17.96.193:45625 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":107,"threads_started":1}
Jun 19 02:30:18 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:18 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:18.272205 26993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:18.263798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.194:43947)
0619 02:30:18.264116 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:18.264134 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:18.264272 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:18.264707 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:18.264710 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:18.264941 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:18.265038 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:18.265044 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:18.266586 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:18.266593 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:18.267824 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:18.267841 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:18.268540 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:18.268544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:18.268626 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:18.269333 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:18.269350 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:18.272017 (+ 2667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":108,"threads_started":1}
W20260619 02:30:18.272421 26992 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:18.263891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:43947 (local address 127.17.96.254:45519)
0619 02:30:18.264092 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:18.264098 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:18.264236 (+ 138us) server_negotiation.cc:407] Connection header received
0619 02:30:18.264429 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:18.264468 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:18.264563 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:18.264688 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:18.265168 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:18.266346 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:18.268070 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:18.268353 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:18.268471 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:18.272263 (+ 3792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.194:43947: BlockingRecv error: recv got EOF from 127.17.96.194:43947 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Jun 19 02:30:18 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:18 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:18.497256 26994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:18.490230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.195:53767)
0619 02:30:18.490564 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:18.490580 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:18.490774 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:18.491328 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:18.491332 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:18.491594 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:18.491718 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:18.491725 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:18.492606 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:18.492610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:18.493346 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:18.493356 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:18.494228 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:18.494232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:18.494337 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:18.494983 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:18.494997 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:18.497108 (+ 2111us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":119,"threads_started":1}
W20260619 02:30:18.497577 26992 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:18.490274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:53767 (local address 127.17.96.254:45519)
0619 02:30:18.490429 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:18.490434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:18.490715 (+ 281us) server_negotiation.cc:407] Connection header received
0619 02:30:18.491056 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:18.491058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:18.491113 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:18.491233 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:18.491852 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:18.492438 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:18.493497 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:18.494044 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:18.494167 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:18.497439 (+ 3272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.195:53767: BlockingRecv error: recv got EOF from 127.17.96.195:53767 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Jun 19 02:30:19 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:19 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:19.054207 26995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:19.044049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.193:38147)
0619 02:30:19.044418 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:19.044435 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:19.044562 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:19.044850 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:19.044852 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:19.045065 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:19.045183 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:19.045190 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:19.046108 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:19.046111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:19.046745 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:19.046755 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:19.047264 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:19.047267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:19.047337 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:19.051385 (+ 4048us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:19.051399 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:19.053983 (+ 2584us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":80,"threads_started":1}
W20260619 02:30:19.054270 26996 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:19.044150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:38147 (local address 127.17.96.254:45519)
0619 02:30:19.044486 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:19.044490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:19.044505 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:30:19.044667 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:19.044670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:19.044720 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:19.044808 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:19.045297 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:19.045993 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:19.046888 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:19.047136 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:19.047239 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:19.054133 (+ 6894us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.193:38147: BlockingRecv error: recv got EOF from 127.17.96.193:38147 (error 108)
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":111,"threads_started":1}
Jun 19 02:30:19 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:19 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:19.280041 26997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:19.273142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.194:36337)
0619 02:30:19.273524 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:19.273538 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:19.273681 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:19.274043 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:19.274046 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:19.274280 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:19.274384 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:19.274390 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:19.275261 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:19.275267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:19.275956 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:19.275966 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:19.276535 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:19.276540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:19.276630 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:19.277368 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:19.277382 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:19.279820 (+ 2438us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":124,"threads_started":1}
W20260619 02:30:19.280130 26996 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:19.273222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:36337 (local address 127.17.96.254:45519)
0619 02:30:19.273416 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:19.273421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:19.273620 (+ 199us) server_negotiation.cc:407] Connection header received
0619 02:30:19.273848 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:19.273852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:19.273909 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:19.274023 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:19.274517 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:19.275128 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:19.276099 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:19.276367 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:19.276519 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:19.279980 (+ 3461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.194:36337: BlockingRecv error: recv got EOF from 127.17.96.194:36337 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Jun 19 02:30:19 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:19 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:19.505295 26998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:19.498212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.195:39209)
0619 02:30:19.498596 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:19.498613 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:19.498809 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:19.499165 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:19.499169 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:19.499421 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:19.499542 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:19.499549 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:19.500455 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:19.500460 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:19.501103 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:19.501113 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:19.501683 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:19.501687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:19.501818 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:19.502584 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:19.502597 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:19.505064 (+ 2467us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":103,"threads_started":1}
W20260619 02:30:19.505354 26996 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:19.498335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:39209 (local address 127.17.96.254:45519)
0619 02:30:19.498530 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:19.498537 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:19.498702 (+ 165us) server_negotiation.cc:407] Connection header received
0619 02:30:19.498956 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:19.498962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:19.499027 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:19.499140 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:19.499693 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:19.500303 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:19.501255 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:19.501521 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:19.501675 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:19.505232 (+ 3557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.195:39209: BlockingRecv error: recv got EOF from 127.17.96.195:39209 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Jun 19 02:30:20 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:20 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:20.062408 26999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:20.055377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.193:56711)
0619 02:30:20.055780 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:20.055810 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:20.055925 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:20.056268 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:20.056270 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:20.056474 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:20.056604 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:20.056610 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:20.057503 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:20.057507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:20.058190 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:20.058199 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:20.058690 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:20.058693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:20.058767 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:20.059442 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:20.059456 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:20.062219 (+ 2763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":102,"threads_started":1}
W20260619 02:30:20.062579 27000 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:20.055509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:56711 (local address 127.17.96.254:45519)
0619 02:30:20.055908 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:20.055913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:20.055934 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:30:20.056032 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:20.056035 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:20.056097 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:20.056212 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:20.056755 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:20.057380 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:20.058320 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:20.058563 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:20.058658 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:20.062403 (+ 3745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.193:56711: BlockingRecv error: recv got EOF from 127.17.96.193:56711 (error 108)
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":137,"threads_started":1}
Jun 19 02:30:20 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:20 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:20.287315 27001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:20.281059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.194:39479)
0619 02:30:20.281334 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:20.281349 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:20.281452 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:20.281706 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:20.281708 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:20.281946 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:20.282048 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:20.282054 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:20.282808 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:20.282811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:20.283418 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:20.283426 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:20.283891 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:20.283894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:20.283993 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:20.284628 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:20.284641 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:20.287064 (+ 2423us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":89,"threads_started":1}
W20260619 02:30:20.287460 27000 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:20.281183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:39479 (local address 127.17.96.254:45519)
0619 02:30:20.281365 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:20.281375 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:20.281406 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:30:20.281543 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:20.281545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:20.281596 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:20.281673 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:20.282159 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:20.282708 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:20.283523 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:20.283745 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:20.283827 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:20.287291 (+ 3464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.194:39479: BlockingRecv error: recv got EOF from 127.17.96.194:39479 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Jun 19 02:30:20 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:20 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:20.512485 27002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:20.506428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.195:39273)
0619 02:30:20.506842 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:20.506858 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:20.506960 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:20.507232 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:20.507234 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:20.507429 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:20.507533 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:20.507538 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:20.508332 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:20.508335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:20.508915 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:20.508923 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:20.509357 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:20.509359 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:20.509421 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:20.510169 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:20.510182 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:20.512292 (+ 2110us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":209,"threads_started":1}
W20260619 02:30:20.512522 27000 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:20.506621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:39273 (local address 127.17.96.254:45519)
0619 02:30:20.506777 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:20.506780 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:20.506934 (+ 154us) server_negotiation.cc:407] Connection header received
0619 02:30:20.507056 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:20.507059 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:20.507108 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:20.507194 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:20.507657 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:20.508230 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:20.509025 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:20.509249 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:20.509333 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:20.512419 (+ 3086us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.195:39273: BlockingRecv error: recv got EOF from 127.17.96.195:39273 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Jun 19 02:30:21 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:21 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:21.069954 27003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:21.063508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.193:60335)
0619 02:30:21.063839 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:21.063856 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:21.063976 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:21.064258 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:21.064260 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:21.064464 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:21.064573 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:21.064579 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:21.065433 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:21.065436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:21.066108 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:21.066118 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:21.066844 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:21.066848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:21.066941 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:21.067569 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:21.067583 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:21.069801 (+ 2218us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":139,"threads_started":1}
W20260619 02:30:21.070183 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:21.063693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:60335 (local address 127.17.96.254:45519)
0619 02:30:21.063982 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:21.063986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:21.064003 (+ 17us) server_negotiation.cc:407] Connection header received
0619 02:30:21.064056 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:21.064059 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:21.064108 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:21.064219 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:21.064704 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:21.065312 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:21.066226 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:21.066486 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:21.066816 (+ 330us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:21.070009 (+ 3193us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.193:60335: BlockingRecv error: recv got EOF from 127.17.96.193:60335 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":114,"threads_started":1}
Jun 19 02:30:21 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:21 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:21.294790 27005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:21.288418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.194:48241)
0619 02:30:21.288804 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:21.288821 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:21.288957 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:21.289384 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:21.289388 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:21.289634 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:21.289831 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:21.289840 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:21.290663 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:21.290669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:21.291252 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:21.291262 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:21.291757 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:21.291762 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:21.291848 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:21.292490 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:21.292503 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:21.294643 (+ 2140us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":136,"threads_started":1}
W20260619 02:30:21.295011 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:21.288495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:48241 (local address 127.17.96.254:45519)
0619 02:30:21.288686 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:21.288716 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:21.288912 (+ 196us) server_negotiation.cc:407] Connection header received
0619 02:30:21.289074 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:21.289078 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:21.289168 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:21.289323 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:21.289973 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:21.290537 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:21.291387 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:21.291626 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:21.291746 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:21.294851 (+ 3105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.194:48241: BlockingRecv error: recv got EOF from 127.17.96.194:48241 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Jun 19 02:30:21 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:21 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:21.520370 27006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:21.513411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.195:51385)
0619 02:30:21.513876 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:21.513900 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:21.514118 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:21.514435 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:21.514438 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:21.514729 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:21.514845 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:21.514852 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:21.515768 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:21.515773 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:21.516510 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:21.516519 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:21.517012 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:21.517016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:21.517095 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:21.518020 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:21.518034 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:21.520183 (+ 2149us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":116,"threads_started":1}
W20260619 02:30:21.520546 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:21.513617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:51385 (local address 127.17.96.254:45519)
0619 02:30:21.513876 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:21.513882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:21.514019 (+ 137us) server_negotiation.cc:407] Connection header received
0619 02:30:21.514238 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:21.514242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:21.514310 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:21.514411 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:21.514972 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:21.515653 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:21.516644 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:21.516885 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:21.516999 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:21.520393 (+ 3394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.195:51385: BlockingRecv error: recv got EOF from 127.17.96.195:51385 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Jun 19 02:30:22 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:22 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:22.078200 27007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:22.071183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.193:55229)
0619 02:30:22.071621 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:22.071638 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:22.071772 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:22.072183 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:22.072187 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:22.072406 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:22.072543 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:22.072550 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:22.073461 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:22.073465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:22.074145 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:22.074156 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:22.074816 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:22.074819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:22.074902 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:22.075626 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:22.075640 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:22.077961 (+ 2321us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":184,"threads_started":1}
W20260619 02:30:22.078473 27008 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:22.071429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:55229 (local address 127.17.96.254:45519)
0619 02:30:22.071762 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:22.071766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:22.071786 (+ 20us) server_negotiation.cc:407] Connection header received
0619 02:30:22.071857 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:22.071859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:22.071917 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:22.072052 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:22.072695 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:22.073296 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:22.074342 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:22.074659 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:22.074786 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:22.078290 (+ 3504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.193:55229: BlockingRecv error: recv got EOF from 127.17.96.193:55229 (error 108)
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":127,"threads_started":1}
Jun 19 02:30:22 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:22 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:22.302268 27009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:22.295841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.194:36403)
0619 02:30:22.296237 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:22.296258 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:22.296421 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:22.296715 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:22.296718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:22.296968 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:22.297067 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:22.297073 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:22.297961 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:22.297967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:22.298554 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:22.298563 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:22.299071 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:22.299074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:22.299152 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:22.299747 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:22.299760 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:22.302067 (+ 2307us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":132,"threads_started":1}
W20260619 02:30:22.302378 27008 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:22.295991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:36403 (local address 127.17.96.254:45519)
0619 02:30:22.296192 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:22.296198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:22.296355 (+ 157us) server_negotiation.cc:407] Connection header received
0619 02:30:22.296537 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:22.296540 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:22.296599 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:22.296692 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:22.297196 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:22.297844 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:22.298698 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:22.298941 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:22.299059 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:22.302242 (+ 3183us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.194:36403: BlockingRecv error: recv got EOF from 127.17.96.194:36403 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Jun 19 02:30:22 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:22 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:22.527841 27010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:22.521469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.195:37135)
0619 02:30:22.521839 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:22.521853 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:22.522024 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:22.522340 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:22.522343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:22.522586 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:22.522694 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:22.522700 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:22.523573 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:22.523580 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:22.524229 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:22.524238 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:22.524768 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:22.524772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:22.524866 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:22.525493 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:22.525506 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:22.527635 (+ 2129us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":109,"threads_started":1}
W20260619 02:30:22.527915 27008 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:22.521529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:37135 (local address 127.17.96.254:45519)
0619 02:30:22.521724 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:22.521729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:22.521938 (+ 209us) server_negotiation.cc:407] Connection header received
0619 02:30:22.522149 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:22.522153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:22.522213 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:22.522315 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:22.522833 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:22.523431 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:22.524372 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:22.524631 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:22.524761 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:22.527791 (+ 3030us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.195:37135: BlockingRecv error: recv got EOF from 127.17.96.195:37135 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Jun 19 02:30:23 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:23 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:23.086349 27011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:23.079527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.193:44949)
0619 02:30:23.079875 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:23.079898 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:23.080025 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:23.080501 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:23.080505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:23.080732 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:23.080870 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:23.080877 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:23.081957 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:23.081962 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:23.082714 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:23.082727 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:23.083293 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:23.083298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:23.083409 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:23.083974 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:23.083986 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:23.086132 (+ 2146us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":124,"threads_started":1}
W20260619 02:30:23.086428 27012 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:23.079619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:44949 (local address 127.17.96.254:45519)
0619 02:30:23.080005 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:23.080009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:23.080032 (+ 23us) server_negotiation.cc:407] Connection header received
0619 02:30:23.080121 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:23.080123 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:23.080176 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:23.080310 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:23.081049 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:23.081737 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:23.082865 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:23.083150 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:23.083273 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:23.086290 (+ 3017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.193:44949: BlockingRecv error: recv got EOF from 127.17.96.193:44949 (error 108)
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":163,"threads_started":1}
Jun 19 02:30:23 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:23 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:23.309845 27013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:23.303400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.194:38669)
0619 02:30:23.303769 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:23.303790 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:23.303929 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:23.304265 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:23.304269 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:23.304509 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:23.304614 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:23.304620 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:23.305503 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:23.305507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:23.306230 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:23.306240 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:23.306763 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:23.306767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:23.306856 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:23.307484 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:23.307497 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:23.309598 (+ 2101us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":110,"threads_started":1}
W20260619 02:30:23.309940 27012 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:23.303627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:38669 (local address 127.17.96.254:45519)
0619 02:30:23.303839 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:23.303846 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:23.303866 (+ 20us) server_negotiation.cc:407] Connection header received
0619 02:30:23.304057 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:23.304060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:23.304129 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:23.304217 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:23.304754 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:23.305361 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:23.306370 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:23.306626 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:23.306757 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:23.309804 (+ 3047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.194:38669: BlockingRecv error: recv got EOF from 127.17.96.194:38669 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Jun 19 02:30:23 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:23 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:23.534644 27014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:23.528890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.195:41907)
0619 02:30:23.529179 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:23.529194 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:23.529308 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:23.529570 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:23.529572 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:23.529822 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:23.529927 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:23.529933 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:23.530743 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:23.530746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:23.531362 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:23.531371 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:23.531827 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:23.531829 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:23.531892 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:23.532426 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:23.532438 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:23.534493 (+ 2055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":97,"threads_started":1}
W20260619 02:30:23.534857 27012 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:23.528974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:41907 (local address 127.17.96.254:45519)
0619 02:30:23.529143 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:23.529147 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:23.529275 (+ 128us) server_negotiation.cc:407] Connection header received
0619 02:30:23.529392 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:23.529394 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:23.529444 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:23.529533 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:23.530045 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:23.530633 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:23.531490 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:23.531711 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:23.531792 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:23.534676 (+ 2884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.195:41907: BlockingRecv error: recv got EOF from 127.17.96.195:41907 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Jun 19 02:30:24 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:24 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:24.097529 27015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:24.087390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.193:47775)
0619 02:30:24.087810 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:24.087826 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:24.087971 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:24.088351 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:24.088354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:24.088591 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:24.088691 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:24.088697 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:24.089877 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:24.089880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:24.090511 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:24.090521 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:24.090998 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:24.091001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:24.091071 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:24.094963 (+ 3892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:24.094980 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:24.097314 (+ 2334us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":125,"threads_started":1}
W20260619 02:30:24.097667 27016 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:24.087466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:47775 (local address 127.17.96.254:45519)
0619 02:30:24.087889 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:24.087896 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:24.087917 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:30:24.088097 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:24.088103 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:24.088194 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:24.088315 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:24.088854 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:24.089722 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:24.090628 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:24.090884 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:24.090973 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:24.097504 (+ 6531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.193:47775: BlockingRecv error: recv got EOF from 127.17.96.193:47775 (error 108)
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":66,"threads_started":1}
Jun 19 02:30:24 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:24 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:24.318243 27017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:24.311211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.194:39981)
0619 02:30:24.311662 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:24.311681 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:24.311830 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:24.312162 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:24.312165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:24.312400 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:24.312559 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:24.312567 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:24.313506 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:24.313511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:24.314341 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:24.314353 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:24.314880 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:24.314884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:24.314988 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:24.315671 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:24.315685 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:24.318074 (+ 2389us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":203,"threads_started":1}
W20260619 02:30:24.318617 27016 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:24.311356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:39981 (local address 127.17.96.254:45519)
0619 02:30:24.311638 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:24.311649 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:24.311850 (+ 201us) server_negotiation.cc:407] Connection header received
0619 02:30:24.311925 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:24.311927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:24.311991 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:24.312094 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:24.312736 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:24.313368 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:24.314468 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:24.314714 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:24.314875 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:24.318333 (+ 3458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.194:39981: BlockingRecv error: recv got EOF from 127.17.96.194:39981 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Jun 19 02:30:24 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:24 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.195@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:24.542706 27018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:24.535747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.195:36499)
0619 02:30:24.536127 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:24.536150 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:24.536344 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:24.536691 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:24.536694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:24.536947 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:24.537048 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:24.537054 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:24.538004 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:24.538011 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:24.538773 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:24.538784 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:24.539414 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:24.539419 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:24.539515 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:24.540200 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:24.540214 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:24.542548 (+ 2334us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":119,"threads_started":1}
W20260619 02:30:24.542935 27016 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:24.535854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:36499 (local address 127.17.96.254:45519)
0619 02:30:24.536122 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:24.536128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:24.536269 (+ 141us) server_negotiation.cc:407] Connection header received
0619 02:30:24.536494 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:24.536498 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:24.536564 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:24.536669 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:24.537182 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:24.537877 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:24.538918 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:24.539262 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:24.539398 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:24.542758 (+ 3360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.195:36499: BlockingRecv error: recv got EOF from 127.17.96.195:36499 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Jun 19 02:30:25 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:25 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.193@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:25.105170 27019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:25.098747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.193:43859)
0619 02:30:25.099088 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:25.099103 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:25.099193 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:25.099523 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:25.099526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:25.099734 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:25.099838 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:25.099843 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:25.100848 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:25.100851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:25.101512 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:25.101522 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:25.102146 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:25.102149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:25.102231 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:25.102798 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:25.102811 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:25.104984 (+ 2173us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":104,"threads_started":1}
W20260619 02:30:25.105300 27020 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:25.098941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:43859 (local address 127.17.96.254:45519)
0619 02:30:25.099217 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:25.099222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:25.099237 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:30:25.099317 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:25.099320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:25.099362 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:25.099451 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:25.100038 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:25.100692 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:25.101683 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:25.102010 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:25.102124 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:25.105155 (+ 3031us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.193:43859: BlockingRecv error: recv got EOF from 127.17.96.193:43859 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":116,"threads_started":1}
Jun 19 02:30:25 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:30:25 dist-test-slave-4qw6 krb5kdc[25962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.96.194@KRBTEST.COM for oryx/127.17.96.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:30:25.326015 27021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:25.319657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45519 (local address 127.17.96.194:59047)
0619 02:30:25.319966 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:25.319989 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:25.320113 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:25.320379 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:25.320381 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:25.320578 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:25.320685 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:25.320690 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:25.321541 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:25.321546 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:25.322370 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:25.322380 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:25.322980 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:25.322983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:25.323093 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:25.323780 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:25.323804 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:25.325867 (+ 2063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.96.254:45519: Server oryx/127.17.96.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":109,"threads_started":1}
W20260619 02:30:25.326151 27020 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:25.319806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:59047 (local address 127.17.96.254:45519)
0619 02:30:25.319985 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:25.319990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:25.320106 (+ 116us) server_negotiation.cc:407] Connection header received
0619 02:30:25.320193 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:25.320196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:25.320241 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:25.320344 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:25.320808 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:25.321414 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:25.322511 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:25.322814 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:25.322950 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:25.326018 (+ 3068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.96.194:59047: BlockingRecv error: recv got EOF from 127.17.96.194:59047 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 02:30:25.407565 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 26545
I20260619 02:30:25.414408 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 26697
I20260619 02:30:25.420710 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 26828
I20260619 02:30:25.426899 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 26475
2026-06-19T02:30:25Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18442 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-19T02:30:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:30:25Z Disabled control of system clock
I20260619 02:30:25.462199 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:45291
--webserver_interface=127.17.96.254
--webserver_port=0
--builtin_ntp_servers=127.17.96.212:42787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:45291
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 02:30:25.587556 27025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:25.587972 27025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:25.588049 27025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:25.593381 27025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:30:25.593519 27025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:25.593550 27025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:25.593582 27025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:30:25.593621 27025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:30:25.600135 27025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:42787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:45291
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:45291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:25.601706 27025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:25.603401 27025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:25.614197 27030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:25.614326 27031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:25.614293 27033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:25.614444 27025 server_base.cc:1034] running on GCE node
I20260619 02:30:25.615352 27025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:25.616669 27025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:25.617890 27025 hybrid_clock.cc:648] HybridClock initialized: now 1781836225617873 us; error 48 us; skew 500 ppm
I20260619 02:30:25.620097 27025 webserver.cc:460] Webserver started at http://127.17.96.254:45447/ using document root <none> and password file <none>
I20260619 02:30:25.620805 27025 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:25.620920 27025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:25.621181 27025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:30:25.623729 27025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "0fb594d2c97243d68d463feeb1fb5c0a"
format_stamp: "Formatted at 2026-06-19 02:30:25 on dist-test-slave-4qw6"
server_key: "a7cd815ead169f0f12292c872abb5fe6"
server_key_iv: "1f5c81ee2a57cbef15800a53e0bde797"
server_key_version: "encryptionkey@0"
I20260619 02:30:25.624444 27025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "0fb594d2c97243d68d463feeb1fb5c0a"
format_stamp: "Formatted at 2026-06-19 02:30:25 on dist-test-slave-4qw6"
server_key: "a7cd815ead169f0f12292c872abb5fe6"
server_key_iv: "1f5c81ee2a57cbef15800a53e0bde797"
server_key_version: "encryptionkey@0"
I20260619 02:30:25.629232 27025 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 02:30:25.632519 27039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:25.634123 27025 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:30:25.634316 27025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "0fb594d2c97243d68d463feeb1fb5c0a"
format_stamp: "Formatted at 2026-06-19 02:30:25 on dist-test-slave-4qw6"
server_key: "a7cd815ead169f0f12292c872abb5fe6"
server_key_iv: "1f5c81ee2a57cbef15800a53e0bde797"
server_key_version: "encryptionkey@0"
I20260619 02:30:25.634482 27025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:25.653334 27025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:25.654692 27025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:25.655045 27025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:25.665697 27025 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:45291
I20260619 02:30:25.665688 27091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:45291 every 8 connection(s)
I20260619 02:30:25.667138 27025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:30:25.670004 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 27025
I20260619 02:30:25.670161 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:30:25.670534 17795 external_mini_cluster.cc:1428] Setting key 8de7ab74873cb525380306ad009175cc
I20260619 02:30:25.673491 27092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:25.682214 27092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fb594d2c97243d68d463feeb1fb5c0a: Bootstrap starting.
I20260619 02:30:25.683060 27095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:25.672343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34632 (local address 127.17.96.254:45291)
0619 02:30:25.673352 (+ 1009us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:25.673377 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:25.673432 (+ 55us) server_negotiation.cc:407] Connection header received
0619 02:30:25.674407 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:25.674431 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:25.674995 (+ 564us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:25.675451 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:25.676017 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:25.677180 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:25.678305 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:25.678925 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:25.679169 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:25.680469 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:25.680475 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:25.680494 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:30:25.681024 (+ 530us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:25.681098 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:30:25.681125 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:25.681293 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:25.681477 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 02:30:25.681909 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":184,"threads_started":1}
I20260619 02:30:25.685640 27092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fb594d2c97243d68d463feeb1fb5c0a: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:25.686724 27092 log.cc:824] T 00000000000000000000000000000000 P 0fb594d2c97243d68d463feeb1fb5c0a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:30:25.689620 27092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fb594d2c97243d68d463feeb1fb5c0a: No bootstrap required, opened a new log
I20260619 02:30:25.693945 27092 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fb594d2c97243d68d463feeb1fb5c0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fb594d2c97243d68d463feeb1fb5c0a" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45291 } }
I20260619 02:30:25.694324 27092 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fb594d2c97243d68d463feeb1fb5c0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:25.694411 27092 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fb594d2c97243d68d463feeb1fb5c0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb594d2c97243d68d463feeb1fb5c0a, State: Initialized, Role: FOLLOWER
I20260619 02:30:25.695030 27092 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fb594d2c97243d68d463feeb1fb5c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fb594d2c97243d68d463feeb1fb5c0a" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45291 } }
I20260619 02:30:25.695237 27092 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fb594d2c97243d68d463feeb1fb5c0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:30:25.695358 27092 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fb594d2c97243d68d463feeb1fb5c0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:30:25.695470 27092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fb594d2c97243d68d463feeb1fb5c0a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:25.696646 27092 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fb594d2c97243d68d463feeb1fb5c0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fb594d2c97243d68d463feeb1fb5c0a" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45291 } }
I20260619 02:30:25.697060 27092 leader_election.cc:302] T 00000000000000000000000000000000 P 0fb594d2c97243d68d463feeb1fb5c0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fb594d2c97243d68d463feeb1fb5c0a; no voters:
I20260619 02:30:25.697486 27092 leader_election.cc:288] T 00000000000000000000000000000000 P 0fb594d2c97243d68d463feeb1fb5c0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:30:25.697674 27097 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fb594d2c97243d68d463feeb1fb5c0a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:30:25.697999 27097 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fb594d2c97243d68d463feeb1fb5c0a [term 1 LEADER]: Becoming Leader. State: Replica: 0fb594d2c97243d68d463feeb1fb5c0a, State: Running, Role: LEADER
I20260619 02:30:25.698359 27097 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fb594d2c97243d68d463feeb1fb5c0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fb594d2c97243d68d463feeb1fb5c0a" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45291 } }
I20260619 02:30:25.699054 27092 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fb594d2c97243d68d463feeb1fb5c0a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:30:25.700186 27098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fb594d2c97243d68d463feeb1fb5c0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fb594d2c97243d68d463feeb1fb5c0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fb594d2c97243d68d463feeb1fb5c0a" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45291 } } }
I20260619 02:30:25.700346 27098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fb594d2c97243d68d463feeb1fb5c0a [sys.catalog]: This master's current role is: LEADER
I20260619 02:30:25.700906 27106 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:30:25.701099 27099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fb594d2c97243d68d463feeb1fb5c0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fb594d2c97243d68d463feeb1fb5c0a. Latest consensus state: current_term: 1 leader_uuid: "0fb594d2c97243d68d463feeb1fb5c0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fb594d2c97243d68d463feeb1fb5c0a" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45291 } } }
I20260619 02:30:25.701373 27099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fb594d2c97243d68d463feeb1fb5c0a [sys.catalog]: This master's current role is: LEADER
I20260619 02:30:25.705379 27106 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:30:25.712806 27106 catalog_manager.cc:1347] Generated new cluster ID: 97d36f74c7b0477691c481fcb49f4e6c
I20260619 02:30:25.712956 27106 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:30:25.738114 27106 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:30:25.739552 27106 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:30:25.756634 27106 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fb594d2c97243d68d463feeb1fb5c0a: Generated new TSK 0
I20260619 02:30:25.758150 27106 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:30:25.770598 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45291
--builtin_ntp_servers=127.17.96.212:42787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 02:30:25.904373 27116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:25.904757 27116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:25.904805 27116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:25.909631 27116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:25.909734 27116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:25.909827 27116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:30:25.915458 27116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:42787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:25.917263 27116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:25.919425 27116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:25.932627 27122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:25.932704 27121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:25.932641 27124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:25.933024 27116 server_base.cc:1034] running on GCE node
I20260619 02:30:25.933655 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:25.934636 27116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:25.935861 27116 hybrid_clock.cc:648] HybridClock initialized: now 1781836225935844 us; error 39 us; skew 500 ppm
I20260619 02:30:25.938449 27116 webserver.cc:460] Webserver started at http://127.17.96.193:37887/ using document root <none> and password file <none>
I20260619 02:30:25.939247 27116 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:25.939361 27116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:25.939664 27116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:30:25.942214 27116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "cdbe8920d97f46189d355c483a23e8f4"
format_stamp: "Formatted at 2026-06-19 02:30:25 on dist-test-slave-4qw6"
server_key: "52a00732098684a010d8f428b3ea2128"
server_key_iv: "fffd766b9dfae5a61c254395200cfa0f"
server_key_version: "encryptionkey@0"
I20260619 02:30:25.942904 27116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "cdbe8920d97f46189d355c483a23e8f4"
format_stamp: "Formatted at 2026-06-19 02:30:25 on dist-test-slave-4qw6"
server_key: "52a00732098684a010d8f428b3ea2128"
server_key_iv: "fffd766b9dfae5a61c254395200cfa0f"
server_key_version: "encryptionkey@0"
I20260619 02:30:25.947356 27116 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 02:30:25.950568 27130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:25.952301 27116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 02:30:25.952513 27116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "cdbe8920d97f46189d355c483a23e8f4"
format_stamp: "Formatted at 2026-06-19 02:30:25 on dist-test-slave-4qw6"
server_key: "52a00732098684a010d8f428b3ea2128"
server_key_iv: "fffd766b9dfae5a61c254395200cfa0f"
server_key_version: "encryptionkey@0"
I20260619 02:30:25.952680 27116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:25.970417 27116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:25.971256 27116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:25.971488 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:25.972209 27116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:30:25.973335 27116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:30:25.973419 27116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:25.973498 27116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:30:25.973554 27116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:25.986626 27116 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:36993
I20260619 02:30:25.986644 27243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:36993 every 8 connection(s)
I20260619 02:30:25.987722 27116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:30:25.989284 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 27116
I20260619 02:30:25.989446 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:30:25.989876 17795 external_mini_cluster.cc:1428] Setting key 788a2d1823acae8a3af2de0299c00b02
I20260619 02:30:25.992515 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45291
--builtin_ntp_servers=127.17.96.212:42787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 02:30:26.002703 27095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:25.990954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:43499 (local address 127.17.96.254:45291)
0619 02:30:25.991146 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:25.991151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:25.992548 (+ 1397us) server_negotiation.cc:407] Connection header received
0619 02:30:25.995371 (+ 2823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:25.995379 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:25.995474 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:25.995624 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:25.997065 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:25.998193 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:25.999751 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:26.000298 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:26.000510 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:26.002065 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:26.002068 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:26.002071 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:30:26.002087 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:26.002110 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:30:26.002120 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:26.002227 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:26.002472 (+ 245us) server_negotiation.cc:299] Negotiation successful
0619 02:30:26.002560 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 02:30:26.004367 27246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:25.991286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45291 (local address 127.17.96.193:43499)
0619 02:30:25.992337 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:25.992413 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:25.994938 (+ 2525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:25.996036 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:25.996049 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:25.996184 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:25.996840 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:25.996858 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:25.998380 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:25.998384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:25.999593 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:25.999608 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:26.000554 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:26.000558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:26.000742 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:26.001594 (+ 852us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:30:26.001600 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:26.001682 (+ 82us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:30:26.001847 (+ 165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:30:26.002253 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:30:26.002268 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:30:26.002279 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 02:30:26.002477 (+ 198us) client_negotiation.cc:240] Negotiation successful
0619 02:30:26.002922 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":182,"threads_started":1}
I20260619 02:30:26.005985 27244 heartbeater.cc:347] Connected to a master server at 127.17.96.254:45291
I20260619 02:30:26.006305 27244 heartbeater.cc:464] Registering TS with master...
I20260619 02:30:26.007082 27244 heartbeater.cc:511] Master 127.17.96.254:45291 requested a full tablet report, sending...
I20260619 02:30:26.009030 27056 ts_manager.cc:194] Registered new tserver with Master: cdbe8920d97f46189d355c483a23e8f4 (127.17.96.193:36993)
I20260619 02:30:26.010412 27056 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.96.193:43499
W20260619 02:30:26.123623 27247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:26.123981 27247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:26.124040 27247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:26.129104 27247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:26.129185 27247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:26.129249 27247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:30:26.135169 27247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:42787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:26.136519 27247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:26.138196 27247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:26.149475 27255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:26.149475 27253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:26.149518 27252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:26.149520 27247 server_base.cc:1034] running on GCE node
I20260619 02:30:26.150238 27247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:26.151057 27247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:26.152271 27247 hybrid_clock.cc:648] HybridClock initialized: now 1781836226152257 us; error 39 us; skew 500 ppm
I20260619 02:30:26.154625 27247 webserver.cc:460] Webserver started at http://127.17.96.194:38315/ using document root <none> and password file <none>
I20260619 02:30:26.155318 27247 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:26.155426 27247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:26.155694 27247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:30:26.158345 27247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "068d5be51efb4c40b3230a1ac606f08b"
format_stamp: "Formatted at 2026-06-19 02:30:26 on dist-test-slave-4qw6"
server_key: "ed78f5729d59335f380ea599a4578855"
server_key_iv: "0c9a7f39d000b8a0f8159620aa4526d6"
server_key_version: "encryptionkey@0"
I20260619 02:30:26.159040 27247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "068d5be51efb4c40b3230a1ac606f08b"
format_stamp: "Formatted at 2026-06-19 02:30:26 on dist-test-slave-4qw6"
server_key: "ed78f5729d59335f380ea599a4578855"
server_key_iv: "0c9a7f39d000b8a0f8159620aa4526d6"
server_key_version: "encryptionkey@0"
I20260619 02:30:26.163534 27247 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 02:30:26.166957 27261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:26.168746 27247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:30:26.168959 27247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "068d5be51efb4c40b3230a1ac606f08b"
format_stamp: "Formatted at 2026-06-19 02:30:26 on dist-test-slave-4qw6"
server_key: "ed78f5729d59335f380ea599a4578855"
server_key_iv: "0c9a7f39d000b8a0f8159620aa4526d6"
server_key_version: "encryptionkey@0"
I20260619 02:30:26.169121 27247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:26.184894 27247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:26.186262 27247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:26.186600 27247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:26.187481 27247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:30:26.188757 27247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:30:26.188845 27247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:26.188933 27247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:30:26.188993 27247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:26.202291 27374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:41469 every 8 connection(s)
I20260619 02:30:26.202265 27247 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:41469
I20260619 02:30:26.203435 27247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:30:26.212073 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 27247
I20260619 02:30:26.212183 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:30:26.212576 17795 external_mini_cluster.cc:1428] Setting key c752df58b773197512248fb38e7da27f
I20260619 02:30:26.215142 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45291
--builtin_ntp_servers=127.17.96.212:42787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 02:30:26.216962 27095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:26.206427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:47327 (local address 127.17.96.254:45291)
0619 02:30:26.206635 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:26.206641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:26.208451 (+ 1810us) server_negotiation.cc:407] Connection header received
0619 02:30:26.210232 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:26.210240 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:26.210329 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:26.210463 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:26.211678 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:26.212440 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:26.214277 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:26.214571 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:26.214724 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:26.216265 (+ 1541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:26.216269 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:26.216272 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:30:26.216288 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:26.216313 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:30:26.216323 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:26.216425 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:26.216698 (+ 273us) server_negotiation.cc:299] Negotiation successful
0619 02:30:26.216807 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 02:30:26.218585 27377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:26.207128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45291 (local address 127.17.96.194:47327)
0619 02:30:26.208146 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:26.208210 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:26.209733 (+ 1523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:26.210673 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:26.210684 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:26.210793 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:26.211415 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:26.211431 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:26.212608 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:26.212615 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:26.214099 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:26.214114 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:26.214783 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:26.214789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:26.215022 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:26.215887 (+ 865us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:30:26.215888 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:26.215941 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:30:26.216032 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:30:26.216432 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:30:26.216446 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:30:26.216454 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 02:30:26.216671 (+ 217us) client_negotiation.cc:240] Negotiation successful
0619 02:30:26.217126 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":197,"threads_started":1}
I20260619 02:30:26.220175 27375 heartbeater.cc:347] Connected to a master server at 127.17.96.254:45291
I20260619 02:30:26.220475 27375 heartbeater.cc:464] Registering TS with master...
I20260619 02:30:26.221127 27375 heartbeater.cc:511] Master 127.17.96.254:45291 requested a full tablet report, sending...
I20260619 02:30:26.222476 27056 ts_manager.cc:194] Registered new tserver with Master: 068d5be51efb4c40b3230a1ac606f08b (127.17.96.194:41469)
I20260619 02:30:26.223148 27056 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.96.194:47327
W20260619 02:30:26.351603 27378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:26.352015 27378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:26.352109 27378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:26.357040 27378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:26.357159 27378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:26.357262 27378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:30:26.363384 27378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:42787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:26.364804 27378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:26.366542 27378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:26.377691 27383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:26.377681 27384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:26.378010 27378 server_base.cc:1034] running on GCE node
W20260619 02:30:26.378129 27386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:26.378835 27378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:26.379585 27378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:26.380771 27378 hybrid_clock.cc:648] HybridClock initialized: now 1781836226380756 us; error 37 us; skew 500 ppm
I20260619 02:30:26.382999 27378 webserver.cc:460] Webserver started at http://127.17.96.195:37359/ using document root <none> and password file <none>
I20260619 02:30:26.383715 27378 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:26.383790 27378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:26.384070 27378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:30:26.386605 27378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "6a6fdff32eb34e2c987e147d934de875"
format_stamp: "Formatted at 2026-06-19 02:30:26 on dist-test-slave-4qw6"
server_key: "84697efae3056722f8ec1204a4447518"
server_key_iv: "d4bbf5ee4b55f99689d8048dcb951026"
server_key_version: "encryptionkey@0"
I20260619 02:30:26.387696 27378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "6a6fdff32eb34e2c987e147d934de875"
format_stamp: "Formatted at 2026-06-19 02:30:26 on dist-test-slave-4qw6"
server_key: "84697efae3056722f8ec1204a4447518"
server_key_iv: "d4bbf5ee4b55f99689d8048dcb951026"
server_key_version: "encryptionkey@0"
I20260619 02:30:26.393392 27378 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 02:30:26.396839 27392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:26.398653 27378 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:30:26.398865 27378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "6a6fdff32eb34e2c987e147d934de875"
format_stamp: "Formatted at 2026-06-19 02:30:26 on dist-test-slave-4qw6"
server_key: "84697efae3056722f8ec1204a4447518"
server_key_iv: "d4bbf5ee4b55f99689d8048dcb951026"
server_key_version: "encryptionkey@0"
I20260619 02:30:26.399029 27378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:26.420598 27378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:26.421517 27378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:26.421875 27378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:26.422591 27378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:30:26.423779 27378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:30:26.423864 27378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:26.423947 27378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:30:26.424010 27378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:26.434135 27378 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:44129
I20260619 02:30:26.434155 27505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:44129 every 8 connection(s)
I20260619 02:30:26.435232 27378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:30:26.444808 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 27378
I20260619 02:30:26.444995 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:30:26.445359 17795 external_mini_cluster.cc:1428] Setting key ae4354d0c92f4d08d2c6382e8e6e5f32
I20260619 02:30:26.447428 27095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:26.438143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:42029 (local address 127.17.96.254:45291)
0619 02:30:26.438359 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:26.438364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:26.439826 (+ 1462us) server_negotiation.cc:407] Connection header received
0619 02:30:26.441476 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:26.441480 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:26.441553 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:26.441681 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:26.442796 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:26.443386 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:26.444838 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:26.445178 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:26.445286 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:26.446854 (+ 1568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:26.446860 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:26.446863 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:30:26.446879 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:26.446904 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:30:26.446915 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:26.447000 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:26.447226 (+ 226us) server_negotiation.cc:299] Negotiation successful
0619 02:30:26.447290 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 02:30:26.448617 27508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:26.438640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45291 (local address 127.17.96.195:42029)
0619 02:30:26.439632 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:26.439686 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:26.441189 (+ 1503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:26.441923 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:26.441931 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:26.442034 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:26.442622 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:26.442636 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:26.443548 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:26.443551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:26.444669 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:26.444684 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:26.445392 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:26.445399 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:26.445637 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:26.446504 (+ 867us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:30:26.446507 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:26.446573 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:30:26.446657 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:30:26.447037 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:30:26.447048 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:30:26.447056 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 02:30:26.447203 (+ 147us) client_negotiation.cc:240] Negotiation successful
0619 02:30:26.447572 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":145,"threads_started":1}
I20260619 02:30:26.449944 27506 heartbeater.cc:347] Connected to a master server at 127.17.96.254:45291
I20260619 02:30:26.450230 27506 heartbeater.cc:464] Registering TS with master...
I20260619 02:30:26.450850 27506 heartbeater.cc:511] Master 127.17.96.254:45291 requested a full tablet report, sending...
I20260619 02:30:26.452051 27056 ts_manager.cc:194] Registered new tserver with Master: 6a6fdff32eb34e2c987e147d934de875 (127.17.96.195:44129)
I20260619 02:30:26.452697 27056 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.96.195:42029
I20260619 02:30:26.460850 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 02:30:26.464550 27095 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:26.463271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34640 (local address 127.17.96.254:45291)
0619 02:30:26.463484 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:26.463491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:26.463767 (+ 276us) server_negotiation.cc:407] Connection header received
0619 02:30:26.463957 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:26.463960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:26.464021 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:26.464123 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:26.464422 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34640: BlockingRecv error: recv got EOF from 127.0.0.1:34640 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260619 02:30:26.464915 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 27116
I20260619 02:30:26.471053 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 27247
I20260619 02:30:26.477094 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 27378
I20260619 02:30:26.484267 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 27025
2026-06-19T02:30:26Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (1056 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-19T02:30:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:30:26Z Disabled control of system clock
I20260619 02:30:26.520718 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:42589
--webserver_interface=127.17.96.254
--webserver_port=0
--builtin_ntp_servers=127.17.96.212:39333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:42589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 02:30:26.657434 27517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:26.657850 27517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:26.657964 27517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:26.662822 27517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:30:26.663147 27517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:26.663261 27517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:26.663286 27517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:30:26.663305 27517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:30:26.670737 27517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:39333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:42589
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:42589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:26.672130 27517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:26.673722 27517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:26.683255 27525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:26.683259 27522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:26.683259 27523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:26.683734 27517 server_base.cc:1034] running on GCE node
I20260619 02:30:26.684427 27517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:26.685828 27517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:26.687039 27517 hybrid_clock.cc:648] HybridClock initialized: now 1781836226687019 us; error 40 us; skew 500 ppm
I20260619 02:30:26.689378 27517 webserver.cc:460] Webserver started at http://127.17.96.254:42949/ using document root <none> and password file <none>
I20260619 02:30:26.690155 27517 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:26.690241 27517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:26.690519 27517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:30:26.692975 27517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "f740cba6e6774c4ca8bef69773c79872"
format_stamp: "Formatted at 2026-06-19 02:30:26 on dist-test-slave-4qw6"
server_key: "c7f97f40c7ceb86043402e355b42d066"
server_key_iv: "c8b5244daca0a4354773175663456dae"
server_key_version: "encryptionkey@0"
I20260619 02:30:26.693634 27517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "f740cba6e6774c4ca8bef69773c79872"
format_stamp: "Formatted at 2026-06-19 02:30:26 on dist-test-slave-4qw6"
server_key: "c7f97f40c7ceb86043402e355b42d066"
server_key_iv: "c8b5244daca0a4354773175663456dae"
server_key_version: "encryptionkey@0"
I20260619 02:30:26.697857 27517 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.002s
I20260619 02:30:26.700877 27531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:26.702787 27517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:30:26.702979 27517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "f740cba6e6774c4ca8bef69773c79872"
format_stamp: "Formatted at 2026-06-19 02:30:26 on dist-test-slave-4qw6"
server_key: "c7f97f40c7ceb86043402e355b42d066"
server_key_iv: "c8b5244daca0a4354773175663456dae"
server_key_version: "encryptionkey@0"
I20260619 02:30:26.703127 27517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:26.708686 27517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:26.709507 27517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:26.709729 27517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:26.717515 27517 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:42589
I20260619 02:30:26.717530 27583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:42589 every 8 connection(s)
I20260619 02:30:26.718590 27517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:30:26.719375 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 27517
I20260619 02:30:26.719511 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:30:26.719838 17795 external_mini_cluster.cc:1428] Setting key edd3556aede4924a696a041f7168fa4c
I20260619 02:30:26.722402 27584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:26.726672 27587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:26.721361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49454 (local address 127.17.96.254:42589)
0619 02:30:26.722673 (+ 1312us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:26.722698 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:26.722737 (+ 39us) server_negotiation.cc:407] Connection header received
0619 02:30:26.723451 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:26.723469 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:26.723857 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:26.724229 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:26.725152 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:26.725157 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:26.725170 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:30:26.725181 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:26.725244 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:30:26.725262 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:26.725438 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:26.725514 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 02:30:26.725861 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":875,"thread_start_us":714,"threads_started":1}
I20260619 02:30:26.729957 27584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f740cba6e6774c4ca8bef69773c79872: Bootstrap starting.
I20260619 02:30:26.732949 27584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f740cba6e6774c4ca8bef69773c79872: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:26.733994 27584 log.cc:824] T 00000000000000000000000000000000 P f740cba6e6774c4ca8bef69773c79872: Log is configured to *not* fsync() on all Append() calls
I20260619 02:30:26.736670 27584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f740cba6e6774c4ca8bef69773c79872: No bootstrap required, opened a new log
I20260619 02:30:26.740967 27584 raft_consensus.cc:348] T 00000000000000000000000000000000 P f740cba6e6774c4ca8bef69773c79872 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f740cba6e6774c4ca8bef69773c79872" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42589 } }
I20260619 02:30:26.741223 27584 raft_consensus.cc:374] T 00000000000000000000000000000000 P f740cba6e6774c4ca8bef69773c79872 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:26.741282 27584 raft_consensus.cc:729] T 00000000000000000000000000000000 P f740cba6e6774c4ca8bef69773c79872 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f740cba6e6774c4ca8bef69773c79872, State: Initialized, Role: FOLLOWER
I20260619 02:30:26.741788 27584 consensus_queue.cc:260] T 00000000000000000000000000000000 P f740cba6e6774c4ca8bef69773c79872 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f740cba6e6774c4ca8bef69773c79872" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42589 } }
I20260619 02:30:26.741940 27584 raft_consensus.cc:388] T 00000000000000000000000000000000 P f740cba6e6774c4ca8bef69773c79872 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:30:26.742136 27584 raft_consensus.cc:482] T 00000000000000000000000000000000 P f740cba6e6774c4ca8bef69773c79872 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:30:26.742308 27584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f740cba6e6774c4ca8bef69773c79872 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:26.743413 27584 raft_consensus.cc:504] T 00000000000000000000000000000000 P f740cba6e6774c4ca8bef69773c79872 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f740cba6e6774c4ca8bef69773c79872" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42589 } }
I20260619 02:30:26.743814 27584 leader_election.cc:302] T 00000000000000000000000000000000 P f740cba6e6774c4ca8bef69773c79872 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f740cba6e6774c4ca8bef69773c79872; no voters:
I20260619 02:30:26.744217 27584 leader_election.cc:288] T 00000000000000000000000000000000 P f740cba6e6774c4ca8bef69773c79872 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:30:26.744405 27589 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f740cba6e6774c4ca8bef69773c79872 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:30:26.744674 27589 raft_consensus.cc:686] T 00000000000000000000000000000000 P f740cba6e6774c4ca8bef69773c79872 [term 1 LEADER]: Becoming Leader. State: Replica: f740cba6e6774c4ca8bef69773c79872, State: Running, Role: LEADER
I20260619 02:30:26.745007 27589 consensus_queue.cc:237] T 00000000000000000000000000000000 P f740cba6e6774c4ca8bef69773c79872 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f740cba6e6774c4ca8bef69773c79872" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42589 } }
I20260619 02:30:26.745612 27584 sys_catalog.cc:564] T 00000000000000000000000000000000 P f740cba6e6774c4ca8bef69773c79872 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:30:26.747383 27590 sys_catalog.cc:455] T 00000000000000000000000000000000 P f740cba6e6774c4ca8bef69773c79872 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f740cba6e6774c4ca8bef69773c79872" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f740cba6e6774c4ca8bef69773c79872" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42589 } } }
I20260619 02:30:26.747571 27590 sys_catalog.cc:458] T 00000000000000000000000000000000 P f740cba6e6774c4ca8bef69773c79872 [sys.catalog]: This master's current role is: LEADER
I20260619 02:30:26.747931 27591 sys_catalog.cc:455] T 00000000000000000000000000000000 P f740cba6e6774c4ca8bef69773c79872 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f740cba6e6774c4ca8bef69773c79872. Latest consensus state: current_term: 1 leader_uuid: "f740cba6e6774c4ca8bef69773c79872" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f740cba6e6774c4ca8bef69773c79872" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42589 } } }
I20260619 02:30:26.748055 27598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:30:26.748059 27591 sys_catalog.cc:458] T 00000000000000000000000000000000 P f740cba6e6774c4ca8bef69773c79872 [sys.catalog]: This master's current role is: LEADER
I20260619 02:30:26.751554 27598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:30:26.759630 27598 catalog_manager.cc:1347] Generated new cluster ID: 08a39a6b1143478c864226a04a312b14
I20260619 02:30:26.759733 27598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:30:26.778540 27598 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:30:26.779002 27598 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:30:26.791790 27598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f740cba6e6774c4ca8bef69773c79872: Generated new TSK 0
I20260619 02:30:26.792773 27598 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:30:26.796949 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:42589
--builtin_ntp_servers=127.17.96.212:39333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 02:30:26.932053 27608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:26.932442 27608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:26.932490 27608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:26.937628 27608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:26.937723 27608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:26.937835 27608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:30:26.944068 27608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:39333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:42589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:26.945384 27608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:26.947165 27608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:26.958775 27616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:26.958781 27613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:26.958777 27614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:26.959072 27608 server_base.cc:1034] running on GCE node
I20260619 02:30:26.959668 27608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:26.960410 27608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:26.961628 27608 hybrid_clock.cc:648] HybridClock initialized: now 1781836226961597 us; error 55 us; skew 500 ppm
I20260619 02:30:26.963903 27608 webserver.cc:460] Webserver started at http://127.17.96.193:33243/ using document root <none> and password file <none>
I20260619 02:30:26.964745 27608 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:26.964859 27608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:26.965128 27608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:30:26.967813 27608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "b610cd82ea4c46df8b592676555c7f64"
format_stamp: "Formatted at 2026-06-19 02:30:26 on dist-test-slave-4qw6"
server_key: "f43a113e74e555e2af7d9499061c6058"
server_key_iv: "2c956216b6ca708c49243a2262ad4d1b"
server_key_version: "encryptionkey@0"
I20260619 02:30:26.968513 27608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "b610cd82ea4c46df8b592676555c7f64"
format_stamp: "Formatted at 2026-06-19 02:30:26 on dist-test-slave-4qw6"
server_key: "f43a113e74e555e2af7d9499061c6058"
server_key_iv: "2c956216b6ca708c49243a2262ad4d1b"
server_key_version: "encryptionkey@0"
I20260619 02:30:26.973104 27608 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260619 02:30:26.976255 27622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:26.977869 27608 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260619 02:30:26.978035 27608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "b610cd82ea4c46df8b592676555c7f64"
format_stamp: "Formatted at 2026-06-19 02:30:26 on dist-test-slave-4qw6"
server_key: "f43a113e74e555e2af7d9499061c6058"
server_key_iv: "2c956216b6ca708c49243a2262ad4d1b"
server_key_version: "encryptionkey@0"
I20260619 02:30:26.978190 27608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:26.983623 27608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:26.984444 27608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:26.984678 27608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:26.985360 27608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:30:26.986531 27608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:30:26.986606 27608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:26.986683 27608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:30:26.986737 27608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:26.998558 27608 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:41415
I20260619 02:30:26.998616 27735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:41415 every 8 connection(s)
I20260619 02:30:26.999706 27608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:30:27.005090 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 27608
I20260619 02:30:27.005244 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:30:27.005568 17795 external_mini_cluster.cc:1428] Setting key de103b145ecf7fc88557beb32c364a72
I20260619 02:30:27.007869 27587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:27.002064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:57789 (local address 127.17.96.254:42589)
0619 02:30:27.002258 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:27.002264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:27.003669 (+ 1405us) server_negotiation.cc:407] Connection header received
0619 02:30:27.005793 (+ 2124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:27.005797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:27.005858 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:27.005956 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:27.007361 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:27.007366 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:27.007370 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:30:27.007385 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:27.007414 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:30:27.007427 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:27.007521 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:27.007710 (+ 189us) server_negotiation.cc:299] Negotiation successful
0619 02:30:27.007753 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 02:30:27.008126 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:42589
--builtin_ntp_servers=127.17.96.212:39333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260619 02:30:27.009318 27738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:27.002564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:42589 (local address 127.17.96.193:57789)
0619 02:30:27.003458 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:27.003518 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:27.005469 (+ 1951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:27.006185 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:27.006194 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:27.006306 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:27.006984 (+ 678us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:30:27.006986 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:27.007033 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:30:27.007112 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:30:27.007541 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:30:27.007550 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:30:27.007556 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:30:27.007762 (+ 206us) client_negotiation.cc:240] Negotiation successful
0619 02:30:27.008050 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":181,"threads_started":1}
I20260619 02:30:27.010743 27736 heartbeater.cc:347] Connected to a master server at 127.17.96.254:42589
I20260619 02:30:27.011022 27736 heartbeater.cc:464] Registering TS with master...
I20260619 02:30:27.011494 27736 heartbeater.cc:511] Master 127.17.96.254:42589 requested a full tablet report, sending...
I20260619 02:30:27.013249 27548 ts_manager.cc:194] Registered new tserver with Master: b610cd82ea4c46df8b592676555c7f64 (127.17.96.193:41415)
W20260619 02:30:27.132791 27739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:27.133247 27739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:27.133353 27739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:27.138175 27739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:27.138329 27739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:27.138460 27739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:30:27.143972 27739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:39333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:42589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:27.145392 27739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:27.147078 27739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:27.158582 27747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:27.158584 27745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:27.158620 27744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:27.158892 27739 server_base.cc:1034] running on GCE node
I20260619 02:30:27.159497 27739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:27.160338 27739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:27.161545 27739 hybrid_clock.cc:648] HybridClock initialized: now 1781836227161513 us; error 60 us; skew 500 ppm
I20260619 02:30:27.163916 27739 webserver.cc:460] Webserver started at http://127.17.96.194:37797/ using document root <none> and password file <none>
I20260619 02:30:27.164603 27739 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:27.164680 27739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:27.164932 27739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:30:27.167459 27739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "e07e7df374ec4c26833e33fcb2e137e0"
format_stamp: "Formatted at 2026-06-19 02:30:27 on dist-test-slave-4qw6"
server_key: "bb520b44ebd4558332d2eb489f19e96d"
server_key_iv: "ecdf9bb847cfc60cd6033913cefa828f"
server_key_version: "encryptionkey@0"
I20260619 02:30:27.168164 27739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "e07e7df374ec4c26833e33fcb2e137e0"
format_stamp: "Formatted at 2026-06-19 02:30:27 on dist-test-slave-4qw6"
server_key: "bb520b44ebd4558332d2eb489f19e96d"
server_key_iv: "ecdf9bb847cfc60cd6033913cefa828f"
server_key_version: "encryptionkey@0"
I20260619 02:30:27.172497 27739 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 02:30:27.176074 27753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:27.177924 27739 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260619 02:30:27.178071 27739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "e07e7df374ec4c26833e33fcb2e137e0"
format_stamp: "Formatted at 2026-06-19 02:30:27 on dist-test-slave-4qw6"
server_key: "bb520b44ebd4558332d2eb489f19e96d"
server_key_iv: "ecdf9bb847cfc60cd6033913cefa828f"
server_key_version: "encryptionkey@0"
I20260619 02:30:27.178253 27739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:27.184511 27739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:27.185284 27739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:27.185516 27739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:27.186281 27739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:30:27.187397 27739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:30:27.187479 27739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:27.187559 27739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:30:27.187605 27739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:27.198381 27739 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:46649
I20260619 02:30:27.198400 27866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:46649 every 8 connection(s)
I20260619 02:30:27.199519 27739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:30:27.206671 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 27739
I20260619 02:30:27.206838 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:30:27.206842 27587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:27.201800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:60331 (local address 127.17.96.254:42589)
0619 02:30:27.202003 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:27.202009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:27.203283 (+ 1274us) server_negotiation.cc:407] Connection header received
0619 02:30:27.204701 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:27.204704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:27.204765 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:27.204865 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:27.206229 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:27.206232 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:27.206235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:30:27.206248 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:27.206269 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:30:27.206277 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:27.206459 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:27.206597 (+ 138us) server_negotiation.cc:299] Negotiation successful
0619 02:30:27.206656 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 02:30:27.207224 17795 external_mini_cluster.cc:1428] Setting key 9178216ec1fe7fa918f8c162b533c347
I20260619 02:30:27.208041 27869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:27.202316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:42589 (local address 127.17.96.194:60331)
0619 02:30:27.203103 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:27.203152 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:27.204443 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:27.205058 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:27.205066 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:27.205167 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:27.205899 (+ 732us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:30:27.205900 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:27.205949 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:30:27.206027 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:30:27.206437 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:30:27.206444 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:30:27.206449 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:30:27.206601 (+ 152us) client_negotiation.cc:240] Negotiation successful
0619 02:30:27.206900 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":147,"threads_started":1}
I20260619 02:30:27.209316 27867 heartbeater.cc:347] Connected to a master server at 127.17.96.254:42589
I20260619 02:30:27.209594 27867 heartbeater.cc:464] Registering TS with master...
I20260619 02:30:27.209681 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:42589
--builtin_ntp_servers=127.17.96.212:39333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260619 02:30:27.210165 27867 heartbeater.cc:511] Master 127.17.96.254:42589 requested a full tablet report, sending...
I20260619 02:30:27.211387 27548 ts_manager.cc:194] Registered new tserver with Master: e07e7df374ec4c26833e33fcb2e137e0 (127.17.96.194:46649)
W20260619 02:30:27.337236 27870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:27.337563 27870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:27.337610 27870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:27.342080 27870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:27.342161 27870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:27.342226 27870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:30:27.347782 27870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:39333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:42589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:27.349072 27870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:27.350708 27870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:27.362380 27876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:27.362488 27878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:27.362473 27875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:27.362812 27870 server_base.cc:1034] running on GCE node
I20260619 02:30:27.363317 27870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:27.364058 27870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:27.365288 27870 hybrid_clock.cc:648] HybridClock initialized: now 1781836227365246 us; error 64 us; skew 500 ppm
I20260619 02:30:27.367761 27870 webserver.cc:460] Webserver started at http://127.17.96.195:36079/ using document root <none> and password file <none>
I20260619 02:30:27.368470 27870 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:27.368575 27870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:27.368822 27870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:30:27.371246 27870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "c5a9e44c0c5b42a2ad212b0263fe2516"
format_stamp: "Formatted at 2026-06-19 02:30:27 on dist-test-slave-4qw6"
server_key: "c0af9fe8624793956cc911289fac68ca"
server_key_iv: "f68c5d562dab4507af096dbac014213d"
server_key_version: "encryptionkey@0"
I20260619 02:30:27.371892 27870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "c5a9e44c0c5b42a2ad212b0263fe2516"
format_stamp: "Formatted at 2026-06-19 02:30:27 on dist-test-slave-4qw6"
server_key: "c0af9fe8624793956cc911289fac68ca"
server_key_iv: "f68c5d562dab4507af096dbac014213d"
server_key_version: "encryptionkey@0"
I20260619 02:30:27.376463 27870 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:30:27.379561 27884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:27.381042 27870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:30:27.381238 27870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "c5a9e44c0c5b42a2ad212b0263fe2516"
format_stamp: "Formatted at 2026-06-19 02:30:27 on dist-test-slave-4qw6"
server_key: "c0af9fe8624793956cc911289fac68ca"
server_key_iv: "f68c5d562dab4507af096dbac014213d"
server_key_version: "encryptionkey@0"
I20260619 02:30:27.381409 27870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:27.387135 27870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:27.387985 27870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:27.388221 27870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:27.388883 27870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:30:27.390079 27870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:30:27.390158 27870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:27.390251 27870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:30:27.390309 27870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:27.401600 27870 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:40133
I20260619 02:30:27.401626 27997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:40133 every 8 connection(s)
I20260619 02:30:27.402757 27870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:30:27.408231 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 27870
I20260619 02:30:27.408413 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:30:27.408761 17795 external_mini_cluster.cc:1428] Setting key ea85b5c2486db9bf46e33b02b58642e0
I20260619 02:30:27.410234 27587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:27.405093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:45885 (local address 127.17.96.254:42589)
0619 02:30:27.405262 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:27.405266 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:27.406478 (+ 1212us) server_negotiation.cc:407] Connection header received
0619 02:30:27.408019 (+ 1541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:27.408023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:27.408073 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:27.408165 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:27.409615 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:27.409620 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:27.409623 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:30:27.409637 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:27.409661 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:30:27.409673 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:27.409796 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:27.410021 (+ 225us) server_negotiation.cc:299] Negotiation successful
0619 02:30:27.410074 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 02:30:27.411414 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:27.405543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:42589 (local address 127.17.96.195:45885)
0619 02:30:27.406305 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:27.406351 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:27.407763 (+ 1412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:27.408398 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:27.408409 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:27.408538 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:27.409278 (+ 740us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:30:27.409280 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:27.409365 (+ 85us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:30:27.409466 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:30:27.409817 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:30:27.409829 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:30:27.409836 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 02:30:27.410042 (+ 206us) client_negotiation.cc:240] Negotiation successful
0619 02:30:27.410397 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":185,"threads_started":1}
I20260619 02:30:27.412649 27998 heartbeater.cc:347] Connected to a master server at 127.17.96.254:42589
I20260619 02:30:27.412945 27998 heartbeater.cc:464] Registering TS with master...
I20260619 02:30:27.413427 27998 heartbeater.cc:511] Master 127.17.96.254:42589 requested a full tablet report, sending...
I20260619 02:30:27.414608 27548 ts_manager.cc:194] Registered new tserver with Master: c5a9e44c0c5b42a2ad212b0263fe2516 (127.17.96.195:40133)
I20260619 02:30:27.424057 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 02:30:27.427667 27587 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:30:27.426547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49468 (local address 127.17.96.254:42589)
0619 02:30:27.426721 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:27.426726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:27.426793 (+ 67us) server_negotiation.cc:407] Connection header received
0619 02:30:27.426850 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:27.426853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:27.426905 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:27.426987 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:27.427468 (+ 481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49468: BlockingRecv error: recv got EOF from 127.0.0.1:49468 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 02:30:27.428097 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 27608
I20260619 02:30:27.434055 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 27739
I20260619 02:30:27.440198 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 27870
I20260619 02:30:27.446954 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 27517
2026-06-19T02:30:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (961 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:30:27 dist-test-slave-4qw6 krb5kdc[28008](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:30:27 dist-test-slave-4qw6 krb5kdc[28008](info): set up 2 sockets
Jun 19 02:30:27 dist-test-slave-4qw6 krb5kdc[28008](info): commencing operation
krb5kdc: starting...
W20260619 02:30:28.567070 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:30:28 dist-test-slave-4qw6 krb5kdc[28008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:30:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:30:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:30:28.709647 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:38701
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:38701
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:30:28.834604 28021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:28.834992 28021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:28.835079 28021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:28.839509 28021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:30:28.839643 28021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:28.839682 28021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:28.839722 28021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:30:28.839761 28021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:30:28.845852 28021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:38701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:38701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:28.847229 28021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:28.848448 28021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:28.858186 28026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:28.858187 28027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:28.858355 28021 server_base.cc:1034] running on GCE node
W20260619 02:30:28.858222 28029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:28.859186 28021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:28.860471 28021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:28.861653 28021 hybrid_clock.cc:648] HybridClock initialized: now 1781836228861633 us; error 40 us; skew 500 ppm
Jun 19 02:30:28 dist-test-slave-4qw6 krb5kdc[28008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836228, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:30:28.864830 28021 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:30:28.866077 28021 webserver.cc:460] Webserver started at http://127.17.96.254:33939/ using document root <none> and password file <none>
I20260619 02:30:28.866771 28021 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:28.866879 28021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:28.867146 28021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:30:28.869568 28021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "78075056038240049ac8867751a30aff"
format_stamp: "Formatted at 2026-06-19 02:30:28 on dist-test-slave-4qw6"
server_key: "fb3dd60e1c3502424a5c9fa6a7f9818f"
server_key_iv: "7b73619d4f456f3011d367d233854893"
server_key_version: "encryptionkey@0"
I20260619 02:30:28.870378 28021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "78075056038240049ac8867751a30aff"
format_stamp: "Formatted at 2026-06-19 02:30:28 on dist-test-slave-4qw6"
server_key: "fb3dd60e1c3502424a5c9fa6a7f9818f"
server_key_iv: "7b73619d4f456f3011d367d233854893"
server_key_version: "encryptionkey@0"
I20260619 02:30:28.875372 28021 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 02:30:28.878649 28036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:28.880475 28021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:30:28.880692 28021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "78075056038240049ac8867751a30aff"
format_stamp: "Formatted at 2026-06-19 02:30:28 on dist-test-slave-4qw6"
server_key: "fb3dd60e1c3502424a5c9fa6a7f9818f"
server_key_iv: "7b73619d4f456f3011d367d233854893"
server_key_version: "encryptionkey@0"
I20260619 02:30:28.880844 28021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:28.900921 28021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:28.904938 28021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:28.905189 28021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:28.912853 28021 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:38701
I20260619 02:30:28.913180 28088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:38701 every 8 connection(s)
I20260619 02:30:28.914028 28021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:30:28.917316 28089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:28.918458 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 28021
I20260619 02:30:28.918617 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:30:28.918942 17795 external_mini_cluster.cc:1428] Setting key d117fc24361f28686076b58c8dd3aba5
I20260619 02:30:28.924949 28089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff: Bootstrap starting.
I20260619 02:30:28.927796 28089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:28.928843 28089 log.cc:824] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff: Log is configured to *not* fsync() on all Append() calls
Jun 19 02:30:28 dist-test-slave-4qw6 krb5kdc[28008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:30:28.931569 28089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff: No bootstrap required, opened a new log
I20260619 02:30:28.936826 28089 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78075056038240049ac8867751a30aff" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38701 } }
I20260619 02:30:28.937117 28089 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:28.937198 28089 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78075056038240049ac8867751a30aff, State: Initialized, Role: FOLLOWER
I20260619 02:30:28.937395 28092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:28.920673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56818 (local address 127.17.96.254:38701)
0619 02:30:28.921505 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:28.921530 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:28.921598 (+ 68us) server_negotiation.cc:407] Connection header received
0619 02:30:28.922579 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:28.922606 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:28.923131 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:28.923660 (+ 529us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:28.924742 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:28.926125 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:28.927147 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:28.927773 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:28.927995 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:28.931544 (+ 3549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:28.931552 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:28.931570 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:30:28.931605 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:28.933556 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:28.934279 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:28.934285 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:28.934291 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:28.934424 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:28.934754 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:28.934756 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:28.934757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:28.935310 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:28.935503 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:28.935963 (+ 460us) server_negotiation.cc:299] Negotiation successful
0619 02:30:28.936352 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":173,"threads_started":1}
I20260619 02:30:28.937850 28089 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78075056038240049ac8867751a30aff" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38701 } }
I20260619 02:30:28.938006 28089 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:30:28.938107 28089 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:30:28.938227 28089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:28.939991 28089 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78075056038240049ac8867751a30aff" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38701 } }
I20260619 02:30:28.940618 28089 leader_election.cc:302] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78075056038240049ac8867751a30aff; no voters:
I20260619 02:30:28.941040 28089 leader_election.cc:288] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:30:28.941313 28094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:30:28.941695 28094 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff [term 1 LEADER]: Becoming Leader. State: Replica: 78075056038240049ac8867751a30aff, State: Running, Role: LEADER
I20260619 02:30:28.942183 28094 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78075056038240049ac8867751a30aff" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38701 } }
I20260619 02:30:28.943060 28089 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:30:28.944392 28095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78075056038240049ac8867751a30aff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78075056038240049ac8867751a30aff" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38701 } } }
I20260619 02:30:28.944587 28095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff [sys.catalog]: This master's current role is: LEADER
I20260619 02:30:28.944903 28096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78075056038240049ac8867751a30aff. Latest consensus state: current_term: 1 leader_uuid: "78075056038240049ac8867751a30aff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78075056038240049ac8867751a30aff" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38701 } } }
I20260619 02:30:28.945024 28096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff [sys.catalog]: This master's current role is: LEADER
I20260619 02:30:28.945179 28103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:30:28.948604 28103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:30:28.955389 28103 catalog_manager.cc:1347] Generated new cluster ID: 177bfff9888b46b782c0302e93753917
I20260619 02:30:28.955502 28103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:30:28.966382 28103 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:30:28.967576 28103 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:30:28.974217 28103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff: Generated new TSK 0
I20260619 02:30:28.975183 28103 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.96.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
I20260619 02:30:29.014806 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:38701
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:30:29.141214 28115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:29.141568 28115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:29.141676 28115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:29.146595 28115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:29.146682 28115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:29.146754 28115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:30:29.152693 28115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:38701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:29.153939 28115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:29.155006 28115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:30:29.166402 28115 server_base.cc:1034] running on GCE node
W20260619 02:30:29.166397 28121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:29.166397 28123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:29.166425 28120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:29.167243 28115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:29.167991 28115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:29.169165 28115 hybrid_clock.cc:648] HybridClock initialized: now 1781836229169137 us; error 46 us; skew 500 ppm
Jun 19 02:30:29 dist-test-slave-4qw6 krb5kdc[28008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836229, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:30:29.177846 28115 init.cc:377] Logged in from keytab as kudu/127.17.96.193@KRBTEST.COM (short username kudu)
I20260619 02:30:29.179106 28115 webserver.cc:460] Webserver started at http://127.17.96.193:38313/ using document root <none> and password file <none>
I20260619 02:30:29.179816 28115 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:29.179935 28115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:29.180225 28115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:30:29.182936 28115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "40601efbf52e4cfeae24cd724c557b63"
format_stamp: "Formatted at 2026-06-19 02:30:29 on dist-test-slave-4qw6"
server_key: "85dd379c211ce25696c6abb67a08362c"
server_key_iv: "e0b4bdf91d1178f3e2158bb94d000be5"
server_key_version: "encryptionkey@0"
I20260619 02:30:29.183696 28115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "40601efbf52e4cfeae24cd724c557b63"
format_stamp: "Formatted at 2026-06-19 02:30:29 on dist-test-slave-4qw6"
server_key: "85dd379c211ce25696c6abb67a08362c"
server_key_iv: "e0b4bdf91d1178f3e2158bb94d000be5"
server_key_version: "encryptionkey@0"
I20260619 02:30:29.188949 28115 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:30:29.192713 28130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:29.194721 28115 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 02:30:29.194921 28115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "40601efbf52e4cfeae24cd724c557b63"
format_stamp: "Formatted at 2026-06-19 02:30:29 on dist-test-slave-4qw6"
server_key: "85dd379c211ce25696c6abb67a08362c"
server_key_iv: "e0b4bdf91d1178f3e2158bb94d000be5"
server_key_version: "encryptionkey@0"
I20260619 02:30:29.195086 28115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:29.217060 28115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:29.221571 28115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:29.221805 28115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:29.222640 28115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:30:29.223745 28115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:30:29.223805 28115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:29.223853 28115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:30:29.223870 28115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:29.235183 28115 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:42399
I20260619 02:30:29.235220 28243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:42399 every 8 connection(s)
I20260619 02:30:29.236301 28115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:30:29.243979 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 28115
I20260619 02:30:29.244153 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:30:29.244612 17795 external_mini_cluster.cc:1428] Setting key aff71db60b36c87cbcec819c50221c06
Jun 19 02:30:29 dist-test-slave-4qw6 krb5kdc[28008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836229, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:30:29.255959 28092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:29.239286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:34875 (local address 127.17.96.254:38701)
0619 02:30:29.239550 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:29.239557 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:29.240642 (+ 1085us) server_negotiation.cc:407] Connection header received
0619 02:30:29.242493 (+ 1851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:29.242497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:29.242558 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:29.242653 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:29.244763 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.245583 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.247403 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.247641 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.247734 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:29.251219 (+ 3485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:29.251229 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:29.251232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:30:29.251262 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:29.252944 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:29.253853 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:29.253856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:29.253857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:29.253944 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:29.254488 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:29.254491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:29.254493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:29.254744 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:29.254834 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:29.255677 (+ 843us) server_negotiation.cc:299] Negotiation successful
0619 02:30:29.255820 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
WARNING: no policy specified for kudu/127.17.96.194@KRBTEST.COM; defaulting to no policy
I20260619 02:30:29.257448 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:29.239739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38701 (local address 127.17.96.193:34875)
0619 02:30:29.240400 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:29.240462 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:29.242148 (+ 1686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:29.242875 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:29.242884 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:29.243500 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:29.244504 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:29.244548 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.245722 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.245726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:29.247228 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:29.247245 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.247823 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.247831 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:29.248121 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:29.248992 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:29.249040 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:29.250964 (+ 1924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:30:29.253115 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:29.253127 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:29.253154 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:29.253658 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:29.254063 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:29.254071 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:29.254076 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:29.254337 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:29.254881 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:30:29.254891 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:30:29.255348 (+ 457us) client_negotiation.cc:769] Sending connection context
0619 02:30:29.255631 (+ 283us) client_negotiation.cc:240] Negotiation successful
0619 02:30:29.256093 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":147,"threads_started":1}
I20260619 02:30:29.259143 28244 heartbeater.cc:347] Connected to a master server at 127.17.96.254:38701
I20260619 02:30:29.259426 28244 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.194@KRBTEST.COM" created.
I20260619 02:30:29.259979 28244 heartbeater.cc:511] Master 127.17.96.254:38701 requested a full tablet report, sending...
I20260619 02:30:29.261898 28053 ts_manager.cc:194] Registered new tserver with Master: 40601efbf52e4cfeae24cd724c557b63 (127.17.96.193:42399)
I20260619 02:30:29.263167 28053 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:34875
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
I20260619 02:30:29.278539 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:38701
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:30:29.406141 28249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:29.406456 28249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:29.406505 28249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:29.410953 28249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:29.411038 28249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:29.411110 28249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:30:29.417007 28249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:38701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:29.418242 28249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:29.419332 28249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:29.430727 28257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:29.430727 28254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:29.430763 28255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:29.430848 28249 server_base.cc:1034] running on GCE node
I20260619 02:30:29.431563 28249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:29.432312 28249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:29.433524 28249 hybrid_clock.cc:648] HybridClock initialized: now 1781836229433503 us; error 36 us; skew 500 ppm
Jun 19 02:30:29 dist-test-slave-4qw6 krb5kdc[28008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836229, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:30:29.436946 28249 init.cc:377] Logged in from keytab as kudu/127.17.96.194@KRBTEST.COM (short username kudu)
I20260619 02:30:29.438057 28249 webserver.cc:460] Webserver started at http://127.17.96.194:41953/ using document root <none> and password file <none>
I20260619 02:30:29.438731 28249 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:29.438835 28249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:29.439105 28249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:30:29.441468 28249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "d95832b6549d466b9790492d92b9acc0"
format_stamp: "Formatted at 2026-06-19 02:30:29 on dist-test-slave-4qw6"
server_key: "1bd5ffe04f4ad0a1d5d16ac45f3905f7"
server_key_iv: "6a04411f3196819d745478cc683ad5fe"
server_key_version: "encryptionkey@0"
I20260619 02:30:29.442196 28249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "d95832b6549d466b9790492d92b9acc0"
format_stamp: "Formatted at 2026-06-19 02:30:29 on dist-test-slave-4qw6"
server_key: "1bd5ffe04f4ad0a1d5d16ac45f3905f7"
server_key_iv: "6a04411f3196819d745478cc683ad5fe"
server_key_version: "encryptionkey@0"
I20260619 02:30:29.447091 28249 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 02:30:29.450232 28264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:29.451718 28249 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:30:29.451897 28249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "d95832b6549d466b9790492d92b9acc0"
format_stamp: "Formatted at 2026-06-19 02:30:29 on dist-test-slave-4qw6"
server_key: "1bd5ffe04f4ad0a1d5d16ac45f3905f7"
server_key_iv: "6a04411f3196819d745478cc683ad5fe"
server_key_version: "encryptionkey@0"
I20260619 02:30:29.452044 28249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:29.470587 28249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:29.474129 28249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:29.474357 28249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:29.475009 28249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:30:29.475893 28249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:30:29.475971 28249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:29.476068 28249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:30:29.476112 28249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:29.485668 28249 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:39699
I20260619 02:30:29.485690 28377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:39699 every 8 connection(s)
I20260619 02:30:29.486852 28249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:30:29.487699 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 28249
I20260619 02:30:29.487829 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:30:29.488174 17795 external_mini_cluster.cc:1428] Setting key 31ffd5ca6560fa8bfffb40ee75132fdd
WARNING: no policy specified for kudu/127.17.96.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.195@KRBTEST.COM" created.
Jun 19 02:30:29 dist-test-slave-4qw6 krb5kdc[28008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836229, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:30:29.506819 28092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:29.490300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:56321 (local address 127.17.96.254:38701)
0619 02:30:29.490479 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:29.490484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:29.491536 (+ 1052us) server_negotiation.cc:407] Connection header received
0619 02:30:29.493172 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:29.493176 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:29.493240 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:29.493365 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:29.494794 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.495346 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.496548 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.496788 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.496905 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:29.501981 (+ 5076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:29.501985 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:29.501987 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:30:29.502028 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:29.503781 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:29.504577 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:29.504580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:29.504581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:29.504678 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:29.505245 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:29.505250 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:29.505253 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:29.505532 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:29.505637 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:29.506534 (+ 897us) server_negotiation.cc:299] Negotiation successful
0619 02:30:29.506673 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 02:30:29.508375 28380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:29.490836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38701 (local address 127.17.96.194:56321)
0619 02:30:29.491372 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:29.491433 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:29.492927 (+ 1494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:29.493565 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:29.493573 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:29.494072 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:29.494620 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:29.494635 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.495492 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.495501 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:29.496406 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:29.496416 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.496954 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.496958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:29.497136 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:29.497953 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:29.497972 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:29.501667 (+ 3695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:30:29.503944 (+ 2277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:29.503955 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:29.503980 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:29.504443 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:29.504814 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:29.504820 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:29.504826 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:29.505080 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:29.505678 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:30:29.505687 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:30:29.506182 (+ 495us) client_negotiation.cc:769] Sending connection context
0619 02:30:29.506483 (+ 301us) client_negotiation.cc:240] Negotiation successful
0619 02:30:29.506979 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":164,"threads_started":1}
I20260619 02:30:29.509992 28378 heartbeater.cc:347] Connected to a master server at 127.17.96.254:38701
I20260619 02:30:29.510363 28378 heartbeater.cc:464] Registering TS with master...
I20260619 02:30:29.511081 28378 heartbeater.cc:511] Master 127.17.96.254:38701 requested a full tablet report, sending...
I20260619 02:30:29.512367 28053 ts_manager.cc:194] Registered new tserver with Master: d95832b6549d466b9790492d92b9acc0 (127.17.96.194:39699)
I20260619 02:30:29.513281 28053 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:56321
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
I20260619 02:30:29.521385 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:38701
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:30:29.649031 28383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:30:29.649408 28383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:30:29.649497 28383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:30:29.654258 28383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:30:29.654392 28383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:30:29.654505 28383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:30:29.661129 28383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:38701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:30:29.662643 28383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:30:29.663815 28383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:30:29.675706 28388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:29.675727 28391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:30:29.675714 28389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:30:29.676059 28383 server_base.cc:1034] running on GCE node
I20260619 02:30:29.676653 28383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:30:29.677503 28383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:30:29.678751 28383 hybrid_clock.cc:648] HybridClock initialized: now 1781836229678739 us; error 62 us; skew 500 ppm
Jun 19 02:30:29 dist-test-slave-4qw6 krb5kdc[28008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836229, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:30:29.682633 28383 init.cc:377] Logged in from keytab as kudu/127.17.96.195@KRBTEST.COM (short username kudu)
I20260619 02:30:29.683684 28383 webserver.cc:460] Webserver started at http://127.17.96.195:38005/ using document root <none> and password file <none>
I20260619 02:30:29.684281 28383 fs_manager.cc:359] Metadata directory not provided
I20260619 02:30:29.684345 28383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:30:29.684598 28383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:30:29.687052 28383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "e45c708640cc497e958809b3dbcb51a8"
format_stamp: "Formatted at 2026-06-19 02:30:29 on dist-test-slave-4qw6"
server_key: "469ee24c31a111b6c89c1026c5286cda"
server_key_iv: "b45f6ac9083ac106bd9a5e088342e7e1"
server_key_version: "encryptionkey@0"
I20260619 02:30:29.687682 28383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "e45c708640cc497e958809b3dbcb51a8"
format_stamp: "Formatted at 2026-06-19 02:30:29 on dist-test-slave-4qw6"
server_key: "469ee24c31a111b6c89c1026c5286cda"
server_key_iv: "b45f6ac9083ac106bd9a5e088342e7e1"
server_key_version: "encryptionkey@0"
I20260619 02:30:29.692013 28383 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:30:29.695422 28398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:29.697679 28383 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260619 02:30:29.697968 28383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "e45c708640cc497e958809b3dbcb51a8"
format_stamp: "Formatted at 2026-06-19 02:30:29 on dist-test-slave-4qw6"
server_key: "469ee24c31a111b6c89c1026c5286cda"
server_key_iv: "b45f6ac9083ac106bd9a5e088342e7e1"
server_key_version: "encryptionkey@0"
I20260619 02:30:29.698109 28383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:30:29.720593 28383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:30:29.725253 28383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:30:29.725512 28383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:30:29.726291 28383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:30:29.727334 28383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:30:29.727387 28383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:29.727432 28383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:30:29.727478 28383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:30:29.739768 28383 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:35809
I20260619 02:30:29.739794 28511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:35809 every 8 connection(s)
I20260619 02:30:29.740828 28383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:30:29.750999 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 28383
I20260619 02:30:29.751128 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:30:29.751556 17795 external_mini_cluster.cc:1428] Setting key 6cb4c8661b8b3b9ce2b63a0cef0246f0
Jun 19 02:30:29 dist-test-slave-4qw6 krb5kdc[28008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836229, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:30:29.759115 28092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:29.743249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:46817 (local address 127.17.96.254:38701)
0619 02:30:29.743406 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:29.743410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:29.744590 (+ 1180us) server_negotiation.cc:407] Connection header received
0619 02:30:29.745656 (+ 1066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:29.745660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:29.745787 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:29.745917 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:29.747304 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.747904 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.749288 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.749553 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.749695 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:29.753832 (+ 4137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:29.753837 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:29.753840 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:30:29.753873 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:29.755515 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:29.756484 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:29.756489 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:29.756497 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:29.756668 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:29.757362 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:29.757365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:29.757367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:29.757674 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:29.757894 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:29.758646 (+ 752us) server_negotiation.cc:299] Negotiation successful
0619 02:30:29.758804 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 02:30:29.760038 28514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:29.743571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38701 (local address 127.17.96.195:46817)
0619 02:30:29.744385 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:29.744435 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:29.745405 (+ 970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:29.746115 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:29.746124 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:29.746555 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:30:29.747085 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:29.747100 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.748072 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.748084 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:29.749125 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:29.749137 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.749774 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.749780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:29.750006 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:29.751346 (+ 1340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:30:29.751366 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:30:29.753604 (+ 2238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:30:29.755689 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:29.755698 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:29.755721 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:29.756226 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:29.756841 (+ 615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:30:29.756848 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:30:29.756850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:30:29.757112 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:30:29.757954 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:30:29.757960 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:30:29.758298 (+ 338us) client_negotiation.cc:769] Sending connection context
0619 02:30:29.758535 (+ 237us) client_negotiation.cc:240] Negotiation successful
0619 02:30:29.758865 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":180,"threads_started":1}
I20260619 02:30:29.762208 28512 heartbeater.cc:347] Connected to a master server at 127.17.96.254:38701
I20260619 02:30:29.762503 28512 heartbeater.cc:464] Registering TS with master...
I20260619 02:30:29.763118 28512 heartbeater.cc:511] Master 127.17.96.254:38701 requested a full tablet report, sending...
I20260619 02:30:29.764714 28053 ts_manager.cc:194] Registered new tserver with Master: e45c708640cc497e958809b3dbcb51a8 (127.17.96.195:35809)
I20260619 02:30:29.765591 28053 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:46817
I20260619 02:30:29.767858 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:30:29.778786 28092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:29.770065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56832 (local address 127.17.96.254:38701)
0619 02:30:29.770241 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:29.770247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:29.770387 (+ 140us) server_negotiation.cc:407] Connection header received
0619 02:30:29.770508 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:29.770510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:29.770556 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:29.770650 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:30:29.771371 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.771917 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.772924 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.773147 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.773244 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:29.774866 (+ 1622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:30:29.774870 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:30:29.774872 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:30:29.774898 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:30:29.776419 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:29.776833 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:29.776836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:29.776837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:29.776912 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:30:29.777328 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:30:29.777338 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:30:29.777339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:30:29.777739 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:30:29.778031 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:29.778354 (+ 323us) server_negotiation.cc:299] Negotiation successful
0619 02:30:29.778565 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 02:30:29.782562 28052 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:30:29.786185 28052 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:30:29.802853 28525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:29.796340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:35809 (local address 127.0.0.1:38100)
0619 02:30:29.797276 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:29.797291 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:29.797384 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:29.798150 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:29.798153 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:29.798170 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:30:29.798309 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:29.798319 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.800082 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.800086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:29.801013 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:29.801024 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.802231 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.802235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:29.802345 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:29.802383 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 02:30:29.802522 (+ 139us) client_negotiation.cc:240] Negotiation successful
0619 02:30:29.802668 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":867,"thread_start_us":94,"threads_started":1}
I20260619 02:30:29.803049 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:29.797355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53466 (local address 127.17.96.193:42399)
0619 02:30:29.797785 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:29.797792 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:29.797815 (+ 23us) server_negotiation.cc:407] Connection header received
0619 02:30:29.797924 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:29.797929 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:29.798114 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:29.798268 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:30:29.798694 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.799845 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.801503 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.802393 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.802507 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:29.802651 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:29.802766 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 02:30:29.802882 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":112,"threads_started":1}
I20260619 02:30:29.802907 28527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:29.796880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:42399 (local address 127.0.0.1:53466)
0619 02:30:29.797583 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:29.797597 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:29.797705 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:29.798418 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:29.798422 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:29.798441 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:30:29.798550 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:29.798557 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.799968 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.799971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:29.801360 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:29.801372 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.802525 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.802530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:29.802637 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:29.802659 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 02:30:29.802735 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 02:30:29.802796 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"mutex_wait_us":61,"thread_start_us":118,"threads_started":1}
I20260619 02:30:29.803814 28526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:29.796835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38100 (local address 127.17.96.195:35809)
0619 02:30:29.797449 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:29.797465 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:29.797484 (+ 19us) server_negotiation.cc:407] Connection header received
0619 02:30:29.797564 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:29.797568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:29.797736 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:29.797944 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:30:29.798445 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.799926 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.801163 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.802070 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.802203 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:29.802371 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:29.803541 (+ 1170us) server_negotiation.cc:299] Negotiation successful
0619 02:30:29.803657 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":109,"threads_started":1}
I20260619 02:30:29.804873 28523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:29.796091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:39699 (local address 127.0.0.1:39962)
0619 02:30:29.796791 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:29.796850 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:29.796998 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:29.797942 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:29.797948 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:29.797973 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:30:29.798126 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:29.798134 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.799714 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.799717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:29.801364 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:29.801372 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.804411 (+ 3039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.804415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:29.804525 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:29.804548 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 02:30:29.804626 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 02:30:29.804730 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":147,"thread_start_us":117,"threads_started":1}
I20260619 02:30:29.804986 28524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:29.796244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39962 (local address 127.17.96.194:39699)
0619 02:30:29.797372 (+ 1128us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:29.797387 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:29.797413 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:30:29.797501 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:29.797508 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:29.797737 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:29.797945 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:30:29.798270 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.799550 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.802464 (+ 2914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.804260 (+ 1796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.804398 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:29.804542 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:29.804706 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 02:30:29.804812 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1013,"thread_start_us":125,"threads_started":1}
I20260619 02:30:29.806859 28446 tablet_service.cc:1467] Processing CreateTablet for tablet 02015d0accfb493781240968a0c8dbcc (DEFAULT_TABLE table=test-table [id=744bd32b305d484db1248a97b9126242]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:30:29.806857 28178 tablet_service.cc:1467] Processing CreateTablet for tablet 02015d0accfb493781240968a0c8dbcc (DEFAULT_TABLE table=test-table [id=744bd32b305d484db1248a97b9126242]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:30:29.808183 28178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02015d0accfb493781240968a0c8dbcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:29.808192 28446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02015d0accfb493781240968a0c8dbcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:29.807868 28312 tablet_service.cc:1467] Processing CreateTablet for tablet 02015d0accfb493781240968a0c8dbcc (DEFAULT_TABLE table=test-table [id=744bd32b305d484db1248a97b9126242]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:30:29.808779 28312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02015d0accfb493781240968a0c8dbcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:30:29.815868 28529 tablet_bootstrap.cc:492] T 02015d0accfb493781240968a0c8dbcc P d95832b6549d466b9790492d92b9acc0: Bootstrap starting.
I20260619 02:30:29.817396 28531 tablet_bootstrap.cc:492] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8: Bootstrap starting.
I20260619 02:30:29.819094 28529 tablet_bootstrap.cc:654] T 02015d0accfb493781240968a0c8dbcc P d95832b6549d466b9790492d92b9acc0: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:29.819446 28531 tablet_bootstrap.cc:654] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:29.819981 28529 log.cc:824] T 02015d0accfb493781240968a0c8dbcc P d95832b6549d466b9790492d92b9acc0: Log is configured to *not* fsync() on all Append() calls
I20260619 02:30:29.820536 28531 log.cc:824] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8: Log is configured to *not* fsync() on all Append() calls
I20260619 02:30:29.820803 28530 tablet_bootstrap.cc:492] T 02015d0accfb493781240968a0c8dbcc P 40601efbf52e4cfeae24cd724c557b63: Bootstrap starting.
I20260619 02:30:29.822386 28529 tablet_bootstrap.cc:492] T 02015d0accfb493781240968a0c8dbcc P d95832b6549d466b9790492d92b9acc0: No bootstrap required, opened a new log
I20260619 02:30:29.822669 28529 ts_tablet_manager.cc:1397] T 02015d0accfb493781240968a0c8dbcc P d95832b6549d466b9790492d92b9acc0: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260619 02:30:29.823135 28530 tablet_bootstrap.cc:654] T 02015d0accfb493781240968a0c8dbcc P 40601efbf52e4cfeae24cd724c557b63: Neither blocks nor log segments found. Creating new log.
I20260619 02:30:29.823746 28531 tablet_bootstrap.cc:492] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8: No bootstrap required, opened a new log
I20260619 02:30:29.824110 28531 ts_tablet_manager.cc:1397] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260619 02:30:29.824156 28530 log.cc:824] T 02015d0accfb493781240968a0c8dbcc P 40601efbf52e4cfeae24cd724c557b63: Log is configured to *not* fsync() on all Append() calls
I20260619 02:30:29.826579 28530 tablet_bootstrap.cc:492] T 02015d0accfb493781240968a0c8dbcc P 40601efbf52e4cfeae24cd724c557b63: No bootstrap required, opened a new log
I20260619 02:30:29.826821 28530 ts_tablet_manager.cc:1397] T 02015d0accfb493781240968a0c8dbcc P 40601efbf52e4cfeae24cd724c557b63: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 02:30:29.827098 28529 raft_consensus.cc:348] T 02015d0accfb493781240968a0c8dbcc P d95832b6549d466b9790492d92b9acc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d95832b6549d466b9790492d92b9acc0" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 39699 } } peers { permanent_uuid: "40601efbf52e4cfeae24cd724c557b63" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 42399 } } peers { permanent_uuid: "e45c708640cc497e958809b3dbcb51a8" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 35809 } }
I20260619 02:30:29.827458 28529 raft_consensus.cc:374] T 02015d0accfb493781240968a0c8dbcc P d95832b6549d466b9790492d92b9acc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:29.827561 28529 raft_consensus.cc:729] T 02015d0accfb493781240968a0c8dbcc P d95832b6549d466b9790492d92b9acc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d95832b6549d466b9790492d92b9acc0, State: Initialized, Role: FOLLOWER
I20260619 02:30:29.828136 28529 consensus_queue.cc:260] T 02015d0accfb493781240968a0c8dbcc P d95832b6549d466b9790492d92b9acc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d95832b6549d466b9790492d92b9acc0" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 39699 } } peers { permanent_uuid: "40601efbf52e4cfeae24cd724c557b63" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 42399 } } peers { permanent_uuid: "e45c708640cc497e958809b3dbcb51a8" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 35809 } }
I20260619 02:30:29.828481 28531 raft_consensus.cc:348] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d95832b6549d466b9790492d92b9acc0" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 39699 } } peers { permanent_uuid: "40601efbf52e4cfeae24cd724c557b63" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 42399 } } peers { permanent_uuid: "e45c708640cc497e958809b3dbcb51a8" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 35809 } }
I20260619 02:30:29.828878 28531 raft_consensus.cc:374] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:29.828982 28531 raft_consensus.cc:729] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e45c708640cc497e958809b3dbcb51a8, State: Initialized, Role: FOLLOWER
I20260619 02:30:29.828996 28378 heartbeater.cc:503] Master 127.17.96.254:38701 was elected leader, sending a full tablet report...
I20260619 02:30:29.829491 28529 ts_tablet_manager.cc:1428] T 02015d0accfb493781240968a0c8dbcc P d95832b6549d466b9790492d92b9acc0: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260619 02:30:29.829581 28531 consensus_queue.cc:260] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d95832b6549d466b9790492d92b9acc0" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 39699 } } peers { permanent_uuid: "40601efbf52e4cfeae24cd724c557b63" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 42399 } } peers { permanent_uuid: "e45c708640cc497e958809b3dbcb51a8" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 35809 } }
I20260619 02:30:29.830523 28512 heartbeater.cc:503] Master 127.17.96.254:38701 was elected leader, sending a full tablet report...
I20260619 02:30:29.830917 28531 ts_tablet_manager.cc:1428] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260619 02:30:29.832072 28530 raft_consensus.cc:348] T 02015d0accfb493781240968a0c8dbcc P 40601efbf52e4cfeae24cd724c557b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d95832b6549d466b9790492d92b9acc0" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 39699 } } peers { permanent_uuid: "40601efbf52e4cfeae24cd724c557b63" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 42399 } } peers { permanent_uuid: "e45c708640cc497e958809b3dbcb51a8" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 35809 } }
I20260619 02:30:29.832336 28530 raft_consensus.cc:374] T 02015d0accfb493781240968a0c8dbcc P 40601efbf52e4cfeae24cd724c557b63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:30:29.832372 28530 raft_consensus.cc:729] T 02015d0accfb493781240968a0c8dbcc P 40601efbf52e4cfeae24cd724c557b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40601efbf52e4cfeae24cd724c557b63, State: Initialized, Role: FOLLOWER
I20260619 02:30:29.832990 28530 consensus_queue.cc:260] T 02015d0accfb493781240968a0c8dbcc P 40601efbf52e4cfeae24cd724c557b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d95832b6549d466b9790492d92b9acc0" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 39699 } } peers { permanent_uuid: "40601efbf52e4cfeae24cd724c557b63" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 42399 } } peers { permanent_uuid: "e45c708640cc497e958809b3dbcb51a8" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 35809 } }
I20260619 02:30:29.834295 28530 ts_tablet_manager.cc:1428] T 02015d0accfb493781240968a0c8dbcc P 40601efbf52e4cfeae24cd724c557b63: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260619 02:30:29.834506 28244 heartbeater.cc:503] Master 127.17.96.254:38701 was elected leader, sending a full tablet report...
I20260619 02:30:29.937269 28536 raft_consensus.cc:482] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:30:29.937510 28536 raft_consensus.cc:504] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d95832b6549d466b9790492d92b9acc0" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 39699 } } peers { permanent_uuid: "40601efbf52e4cfeae24cd724c557b63" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 42399 } } peers { permanent_uuid: "e45c708640cc497e958809b3dbcb51a8" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 35809 } }
I20260619 02:30:29.938803 28536 leader_election.cc:288] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d95832b6549d466b9790492d92b9acc0 (127.17.96.194:39699), 40601efbf52e4cfeae24cd724c557b63 (127.17.96.193:42399)
I20260619 02:30:29.943830 28524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:29.939423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:44421 (local address 127.17.96.194:39699)
0619 02:30:29.939613 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:29.939619 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:29.939643 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:30:29.939996 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:29.940008 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:29.940064 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:29.940206 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:30:29.940533 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.941081 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.942571 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.943018 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.943201 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:29.943305 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:29.943554 (+ 249us) server_negotiation.cc:299] Negotiation successful
0619 02:30:29.943662 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 02:30:29.943872 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:29.939848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:32929 (local address 127.17.96.193:42399)
0619 02:30:29.940000 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:29.940005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:29.940215 (+ 210us) server_negotiation.cc:407] Connection header received
0619 02:30:29.940325 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:29.940328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:29.940382 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:29.940476 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:30:29.940774 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.941270 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.942569 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.943015 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.943168 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:29.943243 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:29.943631 (+ 388us) server_negotiation.cc:299] Negotiation successful
0619 02:30:29.943724 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:30:29.943900 28514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:29.939299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:39699 (local address 127.17.96.195:44421)
0619 02:30:29.939490 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:29.939518 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:29.939653 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:29.940236 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:29.940239 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:29.940260 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:30:29.940401 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:29.940407 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.941217 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.941223 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:29.942423 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:29.942435 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.943173 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.943177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:29.943387 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:29.943438 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 02:30:29.943519 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 02:30:29.943720 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":165}
I20260619 02:30:29.943957 28538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:29.939649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:42399 (local address 127.17.96.195:32929)
0619 02:30:29.940077 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:29.940088 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:30:29.940223 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:30:29.940487 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:30:29.940490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:30:29.940506 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:30:29.940640 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:29.940645 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.941387 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.941392 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:29.942433 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:30:29.942442 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.943161 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.943164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:30:29.943371 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:29.943430 (+ 59us) client_negotiation.cc:769] Sending connection context
0619 02:30:29.943519 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 02:30:29.943752 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":126,"thread_start_us":175,"threads_started":1}
I20260619 02:30:29.944782 28332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02015d0accfb493781240968a0c8dbcc" candidate_uuid: "e45c708640cc497e958809b3dbcb51a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d95832b6549d466b9790492d92b9acc0" is_pre_election: true
I20260619 02:30:29.944788 28198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02015d0accfb493781240968a0c8dbcc" candidate_uuid: "e45c708640cc497e958809b3dbcb51a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40601efbf52e4cfeae24cd724c557b63" is_pre_election: true
I20260619 02:30:29.945178 28332 raft_consensus.cc:2454] T 02015d0accfb493781240968a0c8dbcc P d95832b6549d466b9790492d92b9acc0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e45c708640cc497e958809b3dbcb51a8 in term 0.
I20260619 02:30:29.945178 28198 raft_consensus.cc:2454] T 02015d0accfb493781240968a0c8dbcc P 40601efbf52e4cfeae24cd724c557b63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e45c708640cc497e958809b3dbcb51a8 in term 0.
I20260619 02:30:29.945801 28401 leader_election.cc:302] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d95832b6549d466b9790492d92b9acc0, e45c708640cc497e958809b3dbcb51a8; no voters:
I20260619 02:30:29.946107 28536 raft_consensus.cc:2781] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:30:29.946202 28536 raft_consensus.cc:482] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:30:29.946237 28536 raft_consensus.cc:3037] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:29.947474 28536 raft_consensus.cc:504] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d95832b6549d466b9790492d92b9acc0" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 39699 } } peers { permanent_uuid: "40601efbf52e4cfeae24cd724c557b63" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 42399 } } peers { permanent_uuid: "e45c708640cc497e958809b3dbcb51a8" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 35809 } }
I20260619 02:30:29.947887 28536 leader_election.cc:288] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8 [CANDIDATE]: Term 1 election: Requested vote from peers d95832b6549d466b9790492d92b9acc0 (127.17.96.194:39699), 40601efbf52e4cfeae24cd724c557b63 (127.17.96.193:42399)
I20260619 02:30:29.948441 28332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02015d0accfb493781240968a0c8dbcc" candidate_uuid: "e45c708640cc497e958809b3dbcb51a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d95832b6549d466b9790492d92b9acc0"
I20260619 02:30:29.948441 28198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02015d0accfb493781240968a0c8dbcc" candidate_uuid: "e45c708640cc497e958809b3dbcb51a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40601efbf52e4cfeae24cd724c557b63"
I20260619 02:30:29.948601 28198 raft_consensus.cc:3037] T 02015d0accfb493781240968a0c8dbcc P 40601efbf52e4cfeae24cd724c557b63 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:29.948601 28332 raft_consensus.cc:3037] T 02015d0accfb493781240968a0c8dbcc P d95832b6549d466b9790492d92b9acc0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:30:29.950130 28198 raft_consensus.cc:2454] T 02015d0accfb493781240968a0c8dbcc P 40601efbf52e4cfeae24cd724c557b63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e45c708640cc497e958809b3dbcb51a8 in term 1.
I20260619 02:30:29.950130 28332 raft_consensus.cc:2454] T 02015d0accfb493781240968a0c8dbcc P d95832b6549d466b9790492d92b9acc0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e45c708640cc497e958809b3dbcb51a8 in term 1.
I20260619 02:30:29.950507 28401 leader_election.cc:302] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40601efbf52e4cfeae24cd724c557b63, e45c708640cc497e958809b3dbcb51a8; no voters:
I20260619 02:30:29.950742 28536 raft_consensus.cc:2781] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:30:29.951042 28536 raft_consensus.cc:686] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8 [term 1 LEADER]: Becoming Leader. State: Replica: e45c708640cc497e958809b3dbcb51a8, State: Running, Role: LEADER
I20260619 02:30:29.951460 28536 consensus_queue.cc:237] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d95832b6549d466b9790492d92b9acc0" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 39699 } } peers { permanent_uuid: "40601efbf52e4cfeae24cd724c557b63" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 42399 } } peers { permanent_uuid: "e45c708640cc497e958809b3dbcb51a8" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 35809 } }
I20260619 02:30:29.954963 28052 catalog_manager.cc:5495] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to e45c708640cc497e958809b3dbcb51a8 (127.17.96.195). New cstate: current_term: 1 leader_uuid: "e45c708640cc497e958809b3dbcb51a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d95832b6549d466b9790492d92b9acc0" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 39699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40601efbf52e4cfeae24cd724c557b63" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 42399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e45c708640cc497e958809b3dbcb51a8" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 35809 } health_report { overall_health: HEALTHY } } }
W20260619 02:30:29.988095 28245 tablet.cc:2369] T 02015d0accfb493781240968a0c8dbcc P 40601efbf52e4cfeae24cd724c557b63: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:30:29.988258 28379 tablet.cc:2369] T 02015d0accfb493781240968a0c8dbcc P d95832b6549d466b9790492d92b9acc0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:30:29.991920 28513 tablet.cc:2369] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:30:29.996912 28526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:29.992905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38112 (local address 127.17.96.195:35809)
0619 02:30:29.993063 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 02:30:29.993068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:30:29.993085 (+ 17us) server_negotiation.cc:407] Connection header received
0619 02:30:29.993262 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:30:29.993265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:30:29.993324 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:30:29.993446 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:30:29.993735 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.994318 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.995267 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:30:29.995510 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:30:29.995641 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:30:29.995988 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:30:29.996456 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:30:29.996619 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 02:30:29.996662 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 02:30:29.996727 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 02:30:30.004195 28198 raft_consensus.cc:1261] T 02015d0accfb493781240968a0c8dbcc P 40601efbf52e4cfeae24cd724c557b63 [term 1 FOLLOWER]: Refusing update from remote peer e45c708640cc497e958809b3dbcb51a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:30:30.004195 28332 raft_consensus.cc:1261] T 02015d0accfb493781240968a0c8dbcc P d95832b6549d466b9790492d92b9acc0 [term 1 FOLLOWER]: Refusing update from remote peer e45c708640cc497e958809b3dbcb51a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:30:30.005148 28539 consensus_queue.cc:1035] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40601efbf52e4cfeae24cd724c557b63" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 42399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:30:30.005379 28536 consensus_queue.cc:1035] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d95832b6549d466b9790492d92b9acc0" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 39699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:30:30.015460 28544 mvcc.cc:205] Tried to move back new op lower bound from 7298401198085484544 to 7298401197884547072. Current Snapshot: MvccSnapshot[applied={T|T < 7298401198085484544}]
I20260619 02:30:30.015554 28543 mvcc.cc:205] Tried to move back new op lower bound from 7298401198085484544 to 7298401197884547072. Current Snapshot: MvccSnapshot[applied={T|T < 7298401198085484544}]
I20260619 02:30:30.025353 28053 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:30:30.025547 28053 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:30:30.028513 28053 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 78075056038240049ac8867751a30aff: Sending DeleteTablet for 3 replicas of tablet 02015d0accfb493781240968a0c8dbcc
I20260619 02:30:30.029569 28446 tablet_service.cc:1514] Processing DeleteTablet for tablet 02015d0accfb493781240968a0c8dbcc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:30:30 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:38100
I20260619 02:30:30.029592 28312 tablet_service.cc:1514] Processing DeleteTablet for tablet 02015d0accfb493781240968a0c8dbcc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:30:30 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:39962
I20260619 02:30:30.030212 28553 tablet_replica.cc:330] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8: stopping tablet replica
I20260619 02:30:30.030432 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 28115
I20260619 02:30:30.030584 28553 raft_consensus.cc:2229] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:30:30.030948 28553 raft_consensus.cc:2258] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:30:30.031766 28552 tablet_replica.cc:330] T 02015d0accfb493781240968a0c8dbcc P d95832b6549d466b9790492d92b9acc0: stopping tablet replica
I20260619 02:30:30.032186 28552 raft_consensus.cc:2229] T 02015d0accfb493781240968a0c8dbcc P d95832b6549d466b9790492d92b9acc0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:30:30.032624 28552 raft_consensus.cc:2258] T 02015d0accfb493781240968a0c8dbcc P d95832b6549d466b9790492d92b9acc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:30:30.034489 28552 ts_tablet_manager.cc:1905] T 02015d0accfb493781240968a0c8dbcc P d95832b6549d466b9790492d92b9acc0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:30:30.034489 28553 ts_tablet_manager.cc:1905] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:30:30.037462 28552 ts_tablet_manager.cc:1918] T 02015d0accfb493781240968a0c8dbcc P d95832b6549d466b9790492d92b9acc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:30:30.037462 28553 ts_tablet_manager.cc:1918] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:30:30.037582 28553 log.cc:1196] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/02015d0accfb493781240968a0c8dbcc
I20260619 02:30:30.037778 28552 log.cc:1196] T 02015d0accfb493781240968a0c8dbcc P d95832b6549d466b9790492d92b9acc0: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/02015d0accfb493781240968a0c8dbcc
I20260619 02:30:30.037973 28553 ts_tablet_manager.cc:1939] T 02015d0accfb493781240968a0c8dbcc P e45c708640cc497e958809b3dbcb51a8: Deleting consensus metadata
I20260619 02:30:30.038275 28552 ts_tablet_manager.cc:1939] T 02015d0accfb493781240968a0c8dbcc P d95832b6549d466b9790492d92b9acc0: Deleting consensus metadata
I20260619 02:30:30.039537 28039 catalog_manager.cc:4841] TS d95832b6549d466b9790492d92b9acc0 (127.17.96.194:39699): tablet 02015d0accfb493781240968a0c8dbcc (table test-table [id=744bd32b305d484db1248a97b9126242]) successfully deleted
I20260619 02:30:30.039537 28038 catalog_manager.cc:4841] TS e45c708640cc497e958809b3dbcb51a8 (127.17.96.195:35809): tablet 02015d0accfb493781240968a0c8dbcc (table test-table [id=744bd32b305d484db1248a97b9126242]) successfully deleted
W20260619 02:30:30.040201 28039 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.96.193:42399 (error 108)
I20260619 02:30:30.040490 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 28249
I20260619 02:30:30.042553 28523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:30:30.042222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:42399 (local address 127.0.0.1:53474)
0619 02:30:30.042375 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 02:30:30.042462 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.193:42399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260619 02:30:30.042778 28039 catalog_manager.cc:4584] TS 40601efbf52e4cfeae24cd724c557b63 (127.17.96.193:42399): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 02015d0accfb493781240968a0c8dbcc: Network error: Client connection negotiation failed: client connection to 127.17.96.193:42399: connect: Connection refused (error 111)
I20260619 02:30:30.050062 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 28383
I20260619 02:30:30.056717 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 28021
2026-06-19T02:30:30Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2612 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:30:30 dist-test-slave-4qw6 krb5kdc[28557](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:30:30 dist-test-slave-4qw6 krb5kdc[28557](info): set up 2 sockets
Jun 19 02:30:30 dist-test-slave-4qw6 krb5kdc[28557](info): commencing operation
krb5kdc: starting...
W20260619 02:30:31.195240 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:30:31 dist-test-slave-4qw6 krb5kdc[28557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:30:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:30:31Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.96.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.96.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.96.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/rangeradmin_127.17.96.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.96.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/rangeradmin_127.17.96.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.96.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.96.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.96.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/rangerlookup_127.17.96.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.96.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/rangerlookup_127.17.96.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.96.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.96.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.96.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/HTTP_127.17.96.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.96.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/HTTP_127.17.96.212@KRBTEST.COM.keytab.
I20260619 02:30:31.397048 17795 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskhpFBxW/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/postgres -l logfile start
2026-06-19 02:30:34.172 UTC [28587] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 02:30:34.172 UTC [28587] LOG: listening on IPv4 address "127.17.96.212", port 39559
2026-06-19 02:30:34.176 UTC [28587] LOG: listening on Unix socket "/tmp/.s.PGSQL.39559"
2026-06-19 02:30:34.201 UTC [28590] LOG: database system was shut down at 2026-06-19 02:30:32 UTC
2026-06-19 02:30:34.206 UTC [28587] LOG: database system is ready to accept connections
I20260619 02:30:35.243238 17795 mini_postgres.cc:96] Postgres bound to 39559
2026-06-19 02:30:35.252 UTC [28598] FATAL: database "slave" does not exist
127.17.96.212:39559 - accepting connections
I20260619 02:30:35.252875 17795 mini_ranger.cc:161] Starting Ranger...
I20260619 02:30:35.272248 17795 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 02:30:35.845898 17795 mini_ranger.cc:87] Created ranger Postgres database
I20260619 02:30:35.845989 17795 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ranger-admin
2026-06-19 02:30:36,141 [I] DB FLAVOR :POSTGRES
2026-06-19 02:30:36,142 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 02:30:36,142 [I] Checking connection..
2026-06-19 02:30:36,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:39559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 02:30:36,435 [I] Checking connection passed.
2026-06-19 02:30:36,435 [I] --------- Verifying version history table ---------
2026-06-19 02:30:36,435 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:39559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:30:36,746 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 02:30:36,746 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:39559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:30:37,049 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 02:30:37,049 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 02:30:37,049 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:39559/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 02:30:37.339 UTC [28662] WARNING: there is no transaction in progress
2026-06-19 02:30:37,347 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 02:30:37,347 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:39559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:30:37,641 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 02:30:37,642 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 02:30:37,642 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:39559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 02:30:37,931 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:39559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 02:30:38,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:39559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-4qw6.c.gcp-upstream.internal','N') ;"
2026-06-19 02:30:38,493 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:39559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 02:30:38,777 [I] Table x_portal_user does not exist in database ranger
2026-06-19 02:30:38,778 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:39559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 02:30:39,098 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 02:30:39,098 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:39559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 02:30:39,385 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:39559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 02:30:39,673 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 02:30:39,673 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:39559/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 02:30:40.875 UTC [28770] WARNING: there is no transaction in progress
2026-06-19 02:30:40.894 UTC [28770] WARNING: there is no transaction in progress
2026-06-19 02:30:40.914 UTC [28770] WARNING: there is no transaction in progress
2026-06-19 02:30:40.933 UTC [28770] WARNING: there is no transaction in progress
2026-06-19 02:30:40.952 UTC [28770] WARNING: there is no transaction in progress
2026-06-19 02:30:40.973 UTC [28770] WARNING: there is no transaction in progress
2026-06-19 02:30:41.041 UTC [28770] WARNING: there is no transaction in progress
2026-06-19 02:30:41.050 UTC [28770] WARNING: there is no transaction in progress
2026-06-19 02:30:41.069 UTC [28770] WARNING: there is no transaction in progress
2026-06-19 02:30:41.083 UTC [28770] WARNING: there is no transaction in progress
2026-06-19 02:30:41.096 UTC [28770] WARNING: there is no transaction in progress
2026-06-19 02:30:41.113 UTC [28770] WARNING: there is no transaction in progress
2026-06-19 02:30:41.125 UTC [28770] WARNING: there is no transaction in progress
2026-06-19 02:30:41.133 UTC [28770] WARNING: there is no transaction in progress
2026-06-19 02:30:41.142 UTC [28770] WARNING: there is no transaction in progress
2026-06-19 02:30:41.151 UTC [28770] WARNING: there is no transaction in progress
2026-06-19 02:30:41,975 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 02:30:41,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:39559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 02:30:42,266 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 02:30:42,266 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:39559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 02:30:42,555 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 02:30:42,556 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:39559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 02:30:42,853 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 02:30:42,853 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:39559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 02:30:43,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:39559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 02:30:43,434 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:39559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-4qw6.c.gcp-upstream.internal';"
2026-06-19 02:30:43,716 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 02:30:43,717 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:39559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 02:30:44,034 [I] DB_PATCHES have already been applied
I20260619 02:30:44.040436 17795 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ranger-admin:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskhpFBxW/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 02:30:44.040567 17795 mini_ranger.cc:193] Using host: 127.17.96.212
I20260619 02:30:44.045298 17795 mini_ranger.cc:227] Ranger admin URL: http://127.17.96.212:47687
Jun 19, 2026 2:30:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:30:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:30:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 2:30:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 2:30:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 2:30:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 2:30:45 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 02:30:45.238025 17795 mini_ranger.cc:160] Time spent starting Ranger: real 9.985s user 0.001s sys 0.012s
Jun 19, 2026 2:31:07 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 2:31:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Jun 19, 2026 2:31:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 2:31:07 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 2:31:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 2:31:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 2:31:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 2:31:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 2:31:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 2:31:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 2:31:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 2:31:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 2:31:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 2:31:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 2:31:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 2:31:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 2:31:07 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 2:31:08 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 02:31:08.978018 17795 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.96.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.96.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.96.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/rangerkms_127.17.96.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.96.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/rangerkms_127.17.96.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.96.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.96.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.96.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/HTTP_127.17.96.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.96.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/HTTP_127.17.96.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 02:31:09 dist-test-slave-4qw6 krb5kdc[28557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836269, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 02:31:09.084533 17795 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 02:31:09.097139 17795 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 02:31:10.073371 17795 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 02:31:10.073493 17795 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ranger-kms
I20260619 02:31:10.073612 17795 mini_ranger_kms.cc:227] Using postgres at 127.17.96.212:39559
2026-06-19 02:31:10,351 [I] DB FLAVOR :POSTGRES
2026-06-19 02:31:10,352 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 02:31:10,352 [I] Checking connection
2026-06-19 02:31:10,352 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:39559/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 02:31:10,712 [I] connection success
2026-06-19 02:31:10,712 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 02:31:10,713 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 02:31:10,713 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:39559/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 02:31:11,078 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 02:31:11,079 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 02:31:11,079 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 02:31:11,079 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:39559/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 02:31:11,430 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 02:31:12.070933 17795 mini_ranger_kms.cc:310] Created kms service
I20260619 02:31:12.310328 17795 mini_ranger_kms.cc:326] Created kudu user
I20260619 02:31:12.360597 17795 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 02:31:12 dist-test-slave-4qw6 krb5kdc[28557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836269, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.96.212@KRBTEST.COM
I20260619 02:31:12.869621 17795 mini_ranger_kms.cc:384] Added ranger policy
I20260619 02:31:12.869990 17795 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ranger-kms:/tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 02:31:12.870038 17795 mini_ranger_kms.cc:240] Using host: 127.17.96.212
I20260619 02:31:12.874667 17795 mini_ranger_kms.cc:276] Ranger KMS PID: 28951
I20260619 02:31:12.874817 17795 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.96.212:45183
Jun 19, 2026 2:31:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:31:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:31:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 2:31:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 2:31:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 2:31:15 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 02:31:15.418768 17795 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.334s user 0.003s sys 0.255s
I20260619 02:31:15.418931 17795 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 02:31:15.419073 17795 mini_ranger_kms.cc:401] 127.17.96.212:45183/kms/v1/keys
Jun 19 02:31:15 dist-test-slave-4qw6 krb5kdc[28557](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781836275, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.96.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 02:31:16 dist-test-slave-4qw6 krb5kdc[28557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836275, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.96.212@KRBTEST.COM for HTTP/127.17.96.212@KRBTEST.COM
Jun 19 02:31:16 dist-test-slave-4qw6 krb5kdc[28557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836275, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.96.212@KRBTEST.COM for HTTP/127.17.96.212@KRBTEST.COM
Jun 19 02:31:18 dist-test-slave-4qw6 krb5kdc[28557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:31:18.977253 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:42427
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:37753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:42427
--ranger_config_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.96.212:45183/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:31:19.098188 29001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:31:19.098549 29001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:31:19.098630 29001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:31:19.103757 29001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:31:19.103838 29001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:31:19.103861 29001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:31:19.103880 29001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:31:19.103899 29001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:31:19.110375 29001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:37753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.96.212:45183/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:42427
--ranger_config_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:42427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:31:19.111634 29001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:31:19.112748 29001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:31:19.123001 29009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:31:19.122994 29006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:31:19.123050 29001 server_base.cc:1034] running on GCE node
W20260619 02:31:19.122994 29007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:31:19.123944 29001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:31:19.125212 29001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:31:19.126446 29001 hybrid_clock.cc:648] HybridClock initialized: now 1781836279126404 us; error 61 us; skew 500 ppm
Jun 19 02:31:19 dist-test-slave-4qw6 krb5kdc[28557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836279, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:31:19.134981 29001 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:31:19.136394 29001 webserver.cc:460] Webserver started at http://127.17.96.254:46273/ using document root <none> and password file <none>
I20260619 02:31:19.137223 29001 fs_manager.cc:359] Metadata directory not provided
I20260619 02:31:19.137326 29001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:31:19.137619 29001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:31:19 dist-test-slave-4qw6 krb5kdc[28557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836279, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for HTTP/127.17.96.212@KRBTEST.COM
I20260619 02:31:19.193388 29001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "7cde35ffc93748a2ad65b6a483867cd2"
format_stamp: "Formatted at 2026-06-19 02:31:19 on dist-test-slave-4qw6"
server_key: "66f096664ffad4286d61c1435cc8108b"
server_key_iv: "828ec3b70ade45894faade276235026b"
server_key_version: "kuduclusterkey@0"
I20260619 02:31:19.194305 29001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "7cde35ffc93748a2ad65b6a483867cd2"
format_stamp: "Formatted at 2026-06-19 02:31:19 on dist-test-slave-4qw6"
server_key: "66f096664ffad4286d61c1435cc8108b"
server_key_iv: "828ec3b70ade45894faade276235026b"
server_key_version: "kuduclusterkey@0"
I20260619 02:31:19.200155 29001 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260619 02:31:19.222496 29016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:31:19.224468 29001 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 02:31:19.224645 29001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "7cde35ffc93748a2ad65b6a483867cd2"
format_stamp: "Formatted at 2026-06-19 02:31:19 on dist-test-slave-4qw6"
server_key: "66f096664ffad4286d61c1435cc8108b"
server_key_iv: "828ec3b70ade45894faade276235026b"
server_key_version: "kuduclusterkey@0"
I20260619 02:31:19.224817 29001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:31:19.253901 29001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:31:19.257503 29001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:31:19.257712 29001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:31:19.266729 29001 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:42427
I20260619 02:31:19.266742 29068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:42427 every 8 connection(s)
I20260619 02:31:19.268098 29001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:31:19.271262 29069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:31:19.277145 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 29001
I20260619 02:31:19.277287 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:31:19.277561 17795 external_mini_cluster.cc:1428] Setting key 4cdabc4c65d0fe02474beb6976e23aa1
I20260619 02:31:19.277854 29069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2: Bootstrap starting.
I20260619 02:31:19.281042 29069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2: Neither blocks nor log segments found. Creating new log.
I20260619 02:31:19.281889 29069 log.cc:824] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2: Log is configured to *not* fsync() on all Append() calls
I20260619 02:31:19.284298 29069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2: No bootstrap required, opened a new log
Jun 19 02:31:19 dist-test-slave-4qw6 krb5kdc[28557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:31:19.288719 29069 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cde35ffc93748a2ad65b6a483867cd2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42427 } }
I20260619 02:31:19.289032 29069 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:31:19.289099 29069 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cde35ffc93748a2ad65b6a483867cd2, State: Initialized, Role: FOLLOWER
I20260619 02:31:19.289716 29069 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cde35ffc93748a2ad65b6a483867cd2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42427 } }
I20260619 02:31:19.289954 29069 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:31:19.290042 29069 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:31:19.290153 29069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:31:19.291487 29069 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cde35ffc93748a2ad65b6a483867cd2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42427 } }
I20260619 02:31:19.291890 29069 leader_election.cc:302] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cde35ffc93748a2ad65b6a483867cd2; no voters:
I20260619 02:31:19.292315 29069 leader_election.cc:288] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:31:19.292423 29074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:31:19.292688 29074 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2 [term 1 LEADER]: Becoming Leader. State: Replica: 7cde35ffc93748a2ad65b6a483867cd2, State: Running, Role: LEADER
I20260619 02:31:19.293058 29074 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cde35ffc93748a2ad65b6a483867cd2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42427 } }
I20260619 02:31:19.293845 29069 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:31:19.294582 29074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cde35ffc93748a2ad65b6a483867cd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cde35ffc93748a2ad65b6a483867cd2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42427 } } }
I20260619 02:31:19.294778 29074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:31:19.295023 29074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cde35ffc93748a2ad65b6a483867cd2. Latest consensus state: current_term: 1 leader_uuid: "7cde35ffc93748a2ad65b6a483867cd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cde35ffc93748a2ad65b6a483867cd2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 42427 } } }
I20260619 02:31:19.295159 29074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:31:19.295465 29083 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:31:19.295828 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:31:19.279028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35766 (local address 127.17.96.254:42427)
0619 02:31:19.280005 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0619 02:31:19.280024 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 02:31:19.280071 (+ 47us) server_negotiation.cc:407] Connection header received
0619 02:31:19.281009 (+ 938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:31:19.281034 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:31:19.281467 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:31:19.281913 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:31:19.282762 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:31:19.284161 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:31:19.285370 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:31:19.286185 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:31:19.286363 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:31:19.289551 (+ 3188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:31:19.289557 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:31:19.289575 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:31:19.289612 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:31:19.291717 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:31:19.292257 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:31:19.292266 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:31:19.292273 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:31:19.292473 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:31:19.292807 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:31:19.292810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:31:19.292812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:31:19.293321 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:31:19.294215 (+ 894us) server_negotiation.cc:1035] Waiting for connection context
0619 02:31:19.294373 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 02:31:19.294687 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":797,"thread_start_us":152,"threads_started":1}
I20260619 02:31:19.299149 29083 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:31:19.299638 29069 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 02:31:19.306116 29083 catalog_manager.cc:1347] Generated new cluster ID: 4100cfc672a345efb1a11c800ed6156a
I20260619 02:31:19.306318 29083 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:31:19.322221 29083 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:31:19.324414 29083 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:31:19.336344 29083 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2: Generated new TSK 0
I20260619 02:31:19.337409 29083 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.96.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
I20260619 02:31:20.112547 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.96.212:45183/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:42427
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:37753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:31:20.276183 29114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:31:20.276679 29114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:31:20.276786 29114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:31:20.283406 29114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:31:20.283515 29114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:31:20.283615 29114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:31:20.292310 29114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:37753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.96.212:45183/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:42427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:31:20.294122 29114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:31:20.296124 29114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:31:20.310673 29114 server_base.cc:1034] running on GCE node
W20260619 02:31:20.310940 29122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:31:20.311185 29124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:31:20.311106 29121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:31:20.311832 29114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:31:20.312716 29114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:31:20.313905 29114 hybrid_clock.cc:648] HybridClock initialized: now 1781836280313881 us; error 47 us; skew 500 ppm
Jun 19 02:31:20 dist-test-slave-4qw6 krb5kdc[28557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836280, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:31:20.317588 29114 init.cc:377] Logged in from keytab as kudu/127.17.96.193@KRBTEST.COM (short username kudu)
I20260619 02:31:20.318657 29114 webserver.cc:460] Webserver started at http://127.17.96.193:35623/ using document root <none> and password file <none>
I20260619 02:31:20.319294 29114 fs_manager.cc:359] Metadata directory not provided
I20260619 02:31:20.319360 29114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:31:20.319591 29114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:31:20 dist-test-slave-4qw6 krb5kdc[28557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836280, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for HTTP/127.17.96.212@KRBTEST.COM
I20260619 02:31:20.346488 29114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "3d3155bdbc0e4e63ae8241763f65c14e"
format_stamp: "Formatted at 2026-06-19 02:31:20 on dist-test-slave-4qw6"
server_key: "b616a1499797f92e041502d9016106c7"
server_key_iv: "cdc4f13243cad8f37aa280e6f650690c"
server_key_version: "kuduclusterkey@0"
I20260619 02:31:20.347767 29114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "3d3155bdbc0e4e63ae8241763f65c14e"
format_stamp: "Formatted at 2026-06-19 02:31:20 on dist-test-slave-4qw6"
server_key: "b616a1499797f92e041502d9016106c7"
server_key_iv: "cdc4f13243cad8f37aa280e6f650690c"
server_key_version: "kuduclusterkey@0"
I20260619 02:31:20.353787 29114 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260619 02:31:20.381649 29131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:31:20.395555 29114 fs_manager.cc:725] Time spent opening block manager: real 0.026s user 0.002s sys 0.000s
I20260619 02:31:20.395747 29114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "3d3155bdbc0e4e63ae8241763f65c14e"
format_stamp: "Formatted at 2026-06-19 02:31:20 on dist-test-slave-4qw6"
server_key: "b616a1499797f92e041502d9016106c7"
server_key_iv: "cdc4f13243cad8f37aa280e6f650690c"
server_key_version: "kuduclusterkey@0"
I20260619 02:31:20.395917 29114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:31:20.417551 29114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:31:20.433348 29114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:31:20.433660 29114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:31:20.434598 29114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:31:20.435916 29114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:31:20.436070 29114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:31:20.436218 29114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:31:20.436312 29114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:31:20.448478 29114 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:39417
I20260619 02:31:20.449982 29114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:31:20.452751 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 29114
I20260619 02:31:20.452885 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:31:20.453233 17795 external_mini_cluster.cc:1428] Setting key 9c3c8b63bdbdd3042e3f28f32b4b2ced
I20260619 02:31:20.462599 29246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:39417 every 8 connection(s)
WARNING: no policy specified for kudu/127.17.96.194@KRBTEST.COM; defaulting to no policy
Jun 19 02:31:20 dist-test-slave-4qw6 krb5kdc[28557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836280, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.194@KRBTEST.COM" created.
I20260619 02:31:20.490572 29251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:31:20.465980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:47315 (local address 127.17.96.254:42427)
0619 02:31:20.466895 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0619 02:31:20.466900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:31:20.468745 (+ 1845us) server_negotiation.cc:407] Connection header received
0619 02:31:20.468802 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:31:20.468804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:31:20.468866 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:31:20.468958 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:31:20.470718 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:31:20.471584 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:31:20.473397 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:31:20.473839 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:31:20.473939 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:31:20.485225 (+ 11286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:31:20.485230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:31:20.485233 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:31:20.485271 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:31:20.487400 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:31:20.488288 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:31:20.488293 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:31:20.488294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:31:20.488394 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:31:20.489016 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:31:20.489020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:31:20.489022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:31:20.489285 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:31:20.489377 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 02:31:20.490220 (+ 843us) server_negotiation.cc:299] Negotiation successful
0619 02:31:20.490367 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":823,"thread_start_us":86,"threads_started":1}
I20260619 02:31:20.492650 29250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:31:20.465233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:42427 (local address 127.17.96.193:47315)
0619 02:31:20.467200 (+ 1967us) negotiation.cc:107] Waiting for socket to connect
0619 02:31:20.467274 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0619 02:31:20.468404 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:31:20.469315 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:31:20.469325 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:31:20.469772 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:31:20.470533 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:31:20.470554 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:31:20.471765 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:31:20.471768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:31:20.473238 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:31:20.473251 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:31:20.474185 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:31:20.474190 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:31:20.474401 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:31:20.478576 (+ 4175us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:31:20.478603 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:31:20.484899 (+ 6296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:31:20.487644 (+ 2745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:31:20.487652 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:31:20.487676 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:31:20.488116 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:31:20.488566 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:31:20.488569 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:31:20.488571 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:31:20.488891 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:31:20.489463 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:31:20.489471 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:31:20.489899 (+ 428us) client_negotiation.cc:769] Sending connection context
0619 02:31:20.491089 (+ 1190us) client_negotiation.cc:240] Negotiation successful
0619 02:31:20.491505 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1721,"thread_start_us":169,"threads_started":1}
I20260619 02:31:20.494696 29247 heartbeater.cc:347] Connected to a master server at 127.17.96.254:42427
I20260619 02:31:20.494988 29247 heartbeater.cc:464] Registering TS with master...
I20260619 02:31:20.495654 29247 heartbeater.cc:511] Master 127.17.96.254:42427 requested a full tablet report, sending...
I20260619 02:31:20.497794 29033 ts_manager.cc:194] Registered new tserver with Master: 3d3155bdbc0e4e63ae8241763f65c14e (127.17.96.193:39417)
I20260619 02:31:20.499358 29033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:47315
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
I20260619 02:31:20.515434 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.96.212:45183/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:42427
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:37753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:31:20 dist-test-slave-4qw6 krb5kdc[28557](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781836280, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 02:31:20.718327 29253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:31:20.718869 29253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:31:20.718987 29253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:31:20.726163 29253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:31:20.726347 29253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:31:20.726501 29253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:31:20.734980 29253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:37753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.96.212:45183/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:42427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:31:20.736420 29253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:31:20.737883 29253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:31:20.752230 29262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:31:20.752650 29260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:31:20.752782 29259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:31:20.754354 29253 server_base.cc:1034] running on GCE node
I20260619 02:31:20.754963 29253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:31:20.755995 29253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:31:20.759299 29253 hybrid_clock.cc:648] HybridClock initialized: now 1781836280759152 us; error 156 us; skew 500 ppm
Jun 19 02:31:20 dist-test-slave-4qw6 krb5kdc[28557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836280, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:31:20.763428 29253 init.cc:377] Logged in from keytab as kudu/127.17.96.194@KRBTEST.COM (short username kudu)
I20260619 02:31:20.764797 29253 webserver.cc:460] Webserver started at http://127.17.96.194:32947/ using document root <none> and password file <none>
I20260619 02:31:20.765676 29253 fs_manager.cc:359] Metadata directory not provided
I20260619 02:31:20.765829 29253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:31:20.766156 29253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:31:20 dist-test-slave-4qw6 krb5kdc[28557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836280, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for HTTP/127.17.96.212@KRBTEST.COM
I20260619 02:31:20.784765 29253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "7c7b2eb3ee64456499fac79e31e873b7"
format_stamp: "Formatted at 2026-06-19 02:31:20 on dist-test-slave-4qw6"
server_key: "6bf708e9c46bf7d2cb5591d393968e60"
server_key_iv: "8c6a443e7ce9d235cc112322ab3e2a17"
server_key_version: "kuduclusterkey@0"
I20260619 02:31:20.785615 29253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "7c7b2eb3ee64456499fac79e31e873b7"
format_stamp: "Formatted at 2026-06-19 02:31:20 on dist-test-slave-4qw6"
server_key: "6bf708e9c46bf7d2cb5591d393968e60"
server_key_iv: "8c6a443e7ce9d235cc112322ab3e2a17"
server_key_version: "kuduclusterkey@0"
I20260619 02:31:20.791144 29253 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 02:31:20.819823 29269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:31:20.822363 29253 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260619 02:31:20.822636 29253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "7c7b2eb3ee64456499fac79e31e873b7"
format_stamp: "Formatted at 2026-06-19 02:31:20 on dist-test-slave-4qw6"
server_key: "6bf708e9c46bf7d2cb5591d393968e60"
server_key_iv: "8c6a443e7ce9d235cc112322ab3e2a17"
server_key_version: "kuduclusterkey@0"
I20260619 02:31:20.822883 29253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:31:20.838409 29253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:31:20.841856 29253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:31:20.842077 29253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:31:20.842782 29253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:31:20.843757 29253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:31:20.843832 29253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:31:20.843932 29253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:31:20.843978 29253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:31:20.855463 29253 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:45663
I20260619 02:31:20.855482 29382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:45663 every 8 connection(s)
I20260619 02:31:20.856591 29253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:31:20.864727 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 29253
I20260619 02:31:20.864866 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:31:20.865181 17795 external_mini_cluster.cc:1428] Setting key 41dd22c3ee41ddf8e17fbbf9b9bca44a
Jun 19 02:31:20 dist-test-slave-4qw6 krb5kdc[28557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836280, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.96.195@KRBTEST.COM; defaulting to no policy
I20260619 02:31:20.899261 29251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:31:20.858687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:44167 (local address 127.17.96.254:42427)
0619 02:31:20.859071 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0619 02:31:20.859076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:31:20.859800 (+ 724us) server_negotiation.cc:407] Connection header received
0619 02:31:20.875299 (+ 15499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:31:20.875304 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:31:20.875395 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:31:20.875463 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:31:20.877790 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:31:20.878704 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:31:20.880790 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:31:20.881242 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:31:20.881345 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:31:20.888750 (+ 7405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:31:20.888755 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:31:20.888759 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:31:20.888795 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:31:20.890824 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:31:20.891600 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:31:20.891604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:31:20.891606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:31:20.891713 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:31:20.892166 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:31:20.892170 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:31:20.892172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:31:20.892410 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:31:20.892481 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:31:20.898886 (+ 6405us) server_negotiation.cc:299] Negotiation successful
0619 02:31:20.899074 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265}
I20260619 02:31:20.901542 29385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:31:20.859109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:42427 (local address 127.17.96.194:44167)
0619 02:31:20.859622 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0619 02:31:20.859688 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0619 02:31:20.860649 (+ 961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:31:20.876105 (+ 15456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:31:20.876118 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:31:20.876605 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:31:20.877502 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:31:20.877527 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:31:20.878925 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:31:20.878929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:31:20.880566 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:31:20.880582 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:31:20.881780 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:31:20.881786 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:31:20.882043 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:31:20.886376 (+ 4333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:31:20.886406 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:31:20.888436 (+ 2030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:31:20.891025 (+ 2589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:31:20.891032 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:31:20.891055 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:31:20.891464 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:31:20.891837 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:31:20.891840 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:31:20.891842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:31:20.892057 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:31:20.892568 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:31:20.892576 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:31:20.892974 (+ 398us) client_negotiation.cc:769] Sending connection context
0619 02:31:20.899797 (+ 6823us) client_negotiation.cc:240] Negotiation successful
0619 02:31:20.900272 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":155,"threads_started":1}
I20260619 02:31:20.903427 29383 heartbeater.cc:347] Connected to a master server at 127.17.96.254:42427
I20260619 02:31:20.903715 29383 heartbeater.cc:464] Registering TS with master...
I20260619 02:31:20.904354 29383 heartbeater.cc:511] Master 127.17.96.254:42427 requested a full tablet report, sending...
I20260619 02:31:20.905829 29033 ts_manager.cc:194] Registered new tserver with Master: 7c7b2eb3ee64456499fac79e31e873b7 (127.17.96.194:45663)
I20260619 02:31:20.906623 29033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:44167
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
I20260619 02:31:20.938385 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.96.212:45183/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:42427
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:37753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:31:21 dist-test-slave-4qw6 krb5kdc[28557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836280, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for HTTP/127.17.96.212@KRBTEST.COM
W20260619 02:31:21.120844 29388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:31:21.121307 29388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:31:21.121430 29388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:31:21.128381 29388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:31:21.128549 29388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:31:21.128671 29388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:31:21.137701 29388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:37753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.96.212:45183/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:42427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:31:21.139462 29388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:31:21.140923 29388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:31:21.157600 29393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:31:21.158516 29388 server_base.cc:1034] running on GCE node
W20260619 02:31:21.157601 29394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:31:21.157868 29396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:31:21.159727 29388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:31:21.160571 29388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:31:21.168093 29388 hybrid_clock.cc:648] HybridClock initialized: now 1781836281168036 us; error 80 us; skew 500 ppm
Jun 19 02:31:21 dist-test-slave-4qw6 krb5kdc[28557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836281, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:31:21.171900 29388 init.cc:377] Logged in from keytab as kudu/127.17.96.195@KRBTEST.COM (short username kudu)
I20260619 02:31:21.173213 29388 webserver.cc:460] Webserver started at http://127.17.96.195:45505/ using document root <none> and password file <none>
I20260619 02:31:21.173959 29388 fs_manager.cc:359] Metadata directory not provided
I20260619 02:31:21.174084 29388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:31:21.174352 29388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:31:21 dist-test-slave-4qw6 krb5kdc[28557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836281, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for HTTP/127.17.96.212@KRBTEST.COM
I20260619 02:31:21.188506 29388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "6f7ef65063e84640a81ea1d29e2593c2"
format_stamp: "Formatted at 2026-06-19 02:31:21 on dist-test-slave-4qw6"
server_key: "3bf48d0e4cf37e9a89461df38a533f47"
server_key_iv: "b33a2aade424101ec3fc98c9b5c479e7"
server_key_version: "kuduclusterkey@0"
I20260619 02:31:21.189446 29388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "6f7ef65063e84640a81ea1d29e2593c2"
format_stamp: "Formatted at 2026-06-19 02:31:21 on dist-test-slave-4qw6"
server_key: "3bf48d0e4cf37e9a89461df38a533f47"
server_key_iv: "b33a2aade424101ec3fc98c9b5c479e7"
server_key_version: "kuduclusterkey@0"
I20260619 02:31:21.198912 29388 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.000s sys 0.003s
I20260619 02:31:21.219636 29403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:31:21.221549 29388 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260619 02:31:21.221719 29388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "6f7ef65063e84640a81ea1d29e2593c2"
format_stamp: "Formatted at 2026-06-19 02:31:21 on dist-test-slave-4qw6"
server_key: "3bf48d0e4cf37e9a89461df38a533f47"
server_key_iv: "b33a2aade424101ec3fc98c9b5c479e7"
server_key_version: "kuduclusterkey@0"
I20260619 02:31:21.221915 29388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:31:21.255503 29388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:31:21.258952 29388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:31:21.259240 29388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:31:21.260109 29388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:31:21.261198 29388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:31:21.261276 29388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:31:21.261384 29388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:31:21.261432 29388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:31:21.273229 29388 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:33007
I20260619 02:31:21.276161 29388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:31:21.278578 29516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:33007 every 8 connection(s)
I20260619 02:31:21.285312 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 29388
I20260619 02:31:21.285485 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:31:21.285825 17795 external_mini_cluster.cc:1428] Setting key 11dea72466d954b0a36c37d9a079156d
Jun 19 02:31:21 dist-test-slave-4qw6 krb5kdc[28557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836281, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:31:21.294232 29251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:31:21.278520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:55843 (local address 127.17.96.254:42427)
0619 02:31:21.278679 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 02:31:21.278686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:31:21.279851 (+ 1165us) server_negotiation.cc:407] Connection header received
0619 02:31:21.281016 (+ 1165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:31:21.281020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:31:21.281095 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:31:21.281177 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:31:21.283173 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.283738 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.285855 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.286201 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.286330 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:31:21.289742 (+ 3412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:31:21.289778 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:31:21.289784 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:31:21.289821 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:31:21.291586 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:31:21.292263 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:31:21.292270 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:31:21.292272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:31:21.292367 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:31:21.292799 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:31:21.292804 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:31:21.292806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:31:21.293042 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:31:21.293170 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 02:31:21.293832 (+ 662us) server_negotiation.cc:299] Negotiation successful
0619 02:31:21.294003 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:31:21.295634 29519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:31:21.279070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:42427 (local address 127.17.96.195:55843)
0619 02:31:21.279657 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0619 02:31:21.279709 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 02:31:21.280750 (+ 1041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:31:21.281372 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:31:21.281380 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:31:21.282140 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:31:21.282873 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:31:21.282891 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.283898 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.283903 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:31:21.284877 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:31:21.284889 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.286563 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.286568 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:31:21.286843 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:31:21.287668 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:31:21.287693 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:31:21.289530 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:31:21.291752 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:31:21.291761 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:31:21.291783 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:31:21.292131 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:31:21.292486 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:31:21.292490 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:31:21.292492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:31:21.292682 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:31:21.293170 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:31:21.293181 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:31:21.293514 (+ 333us) client_negotiation.cc:769] Sending connection context
0619 02:31:21.293754 (+ 240us) client_negotiation.cc:240] Negotiation successful
0619 02:31:21.294177 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":162,"threads_started":1}
I20260619 02:31:21.297122 29517 heartbeater.cc:347] Connected to a master server at 127.17.96.254:42427
I20260619 02:31:21.297425 29517 heartbeater.cc:464] Registering TS with master...
I20260619 02:31:21.298272 29517 heartbeater.cc:511] Master 127.17.96.254:42427 requested a full tablet report, sending...
I20260619 02:31:21.299749 29033 ts_manager.cc:194] Registered new tserver with Master: 6f7ef65063e84640a81ea1d29e2593c2 (127.17.96.195:33007)
I20260619 02:31:21.300496 29033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:55843
I20260619 02:31:21.301929 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:31:21.314863 29251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:31:21.305094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35778 (local address 127.17.96.254:42427)
0619 02:31:21.305234 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 02:31:21.305240 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:31:21.305526 (+ 286us) server_negotiation.cc:407] Connection header received
0619 02:31:21.305697 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:31:21.305701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:31:21.305811 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:31:21.305930 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:31:21.306984 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.307574 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.308547 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.308794 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.308931 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:31:21.311161 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:31:21.311165 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:31:21.311167 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:31:21.311198 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:31:21.312882 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:31:21.313411 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:31:21.313416 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:31:21.313418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:31:21.313519 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:31:21.313865 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:31:21.313869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:31:21.313877 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:31:21.314090 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:31:21.314248 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 02:31:21.314528 (+ 280us) server_negotiation.cc:299] Negotiation successful
0619 02:31:21.314663 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 02:31:21.318518 29033 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35778:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:31:21.322991 29033 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:31:21.342847 29529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:31:21.334155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:45663 (local address 127.0.0.1:42932)
0619 02:31:21.336366 (+ 2211us) negotiation.cc:107] Waiting for socket to connect
0619 02:31:21.336413 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 02:31:21.336561 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:31:21.338509 (+ 1948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:31:21.338516 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:31:21.338541 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:31:21.338714 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:31:21.338721 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.339836 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.339842 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:31:21.341173 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:31:21.341182 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.342184 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.342188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:31:21.342316 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:31:21.342358 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 02:31:21.342533 (+ 175us) client_negotiation.cc:240] Negotiation successful
0619 02:31:21.342678 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2054,"thread_start_us":144,"threads_started":1}
I20260619 02:31:21.342860 29530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:31:21.334399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42932 (local address 127.17.96.194:45663)
0619 02:31:21.338000 (+ 3601us) server_negotiation.cc:206] Beginning negotiation
0619 02:31:21.338017 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 02:31:21.338041 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:31:21.338115 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:31:21.338120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:31:21.338309 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:31:21.338506 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:31:21.338840 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.339706 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.341336 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.342040 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.342185 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:31:21.342368 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0619 02:31:21.342570 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 02:31:21.342684 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3487,"thread_start_us":83,"threads_started":1}
I20260619 02:31:21.347108 29303 tablet_service.cc:1467] Processing CreateTablet for tablet 2c45107d61434583a568573b7a2d3e4c (DEFAULT_TABLE table=test-table [id=c4f4b99742744e3ab40e269d94fc0e4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:31:21.348475 29303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c45107d61434583a568573b7a2d3e4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:31:21.350139 29531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:31:21.335374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:33007 (local address 127.0.0.1:36336)
0619 02:31:21.336815 (+ 1441us) negotiation.cc:107] Waiting for socket to connect
0619 02:31:21.336831 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:31:21.336917 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:31:21.343277 (+ 6360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:31:21.343280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:31:21.343298 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:31:21.343375 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:31:21.343381 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.345493 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.345499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:31:21.346621 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:31:21.346636 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.348273 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.348283 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:31:21.348393 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:31:21.348435 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 02:31:21.348653 (+ 218us) client_negotiation.cc:240] Negotiation successful
0619 02:31:21.348756 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1373,"thread_start_us":101,"threads_started":1}
I20260619 02:31:21.351039 29533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:31:21.335806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36336 (local address 127.17.96.195:33007)
0619 02:31:21.342359 (+ 6553us) server_negotiation.cc:206] Beginning negotiation
0619 02:31:21.342371 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:31:21.342398 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:31:21.342482 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:31:21.342488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:31:21.342746 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:31:21.342912 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:31:21.343525 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.345355 (+ 1830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.346834 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.347857 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.347995 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:31:21.348171 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0619 02:31:21.350718 (+ 2547us) server_negotiation.cc:299] Negotiation successful
0619 02:31:21.350862 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6423,"thread_start_us":167,"threads_started":1}
I20260619 02:31:21.359465 29532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:31:21.335717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:39417 (local address 127.0.0.1:55120)
0619 02:31:21.341945 (+ 6228us) negotiation.cc:107] Waiting for socket to connect
0619 02:31:21.341962 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:31:21.342066 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:31:21.354577 (+ 12511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:31:21.354580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:31:21.354609 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:31:21.354756 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:31:21.354764 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.356409 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.356413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:31:21.357637 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:31:21.357651 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.358999 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.359004 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:31:21.359104 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:31:21.359138 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 02:31:21.359200 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 02:31:21.359293 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6142,"thread_start_us":114,"threads_started":1}
I20260619 02:31:21.360224 29535 tablet_bootstrap.cc:492] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7: Bootstrap starting.
I20260619 02:31:21.360642 29534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:31:21.335885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55120 (local address 127.17.96.193:39417)
0619 02:31:21.354007 (+ 18122us) server_negotiation.cc:206] Beginning negotiation
0619 02:31:21.354017 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:31:21.354050 (+ 33us) server_negotiation.cc:407] Connection header received
0619 02:31:21.354135 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:31:21.354140 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:31:21.354358 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:31:21.354933 (+ 575us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:31:21.355080 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.356239 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.357855 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.358827 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.360033 (+ 1206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:31:21.360208 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0619 02:31:21.360288 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 02:31:21.360491 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17986,"thread_start_us":114,"threads_started":1}
I20260619 02:31:21.364228 29174 tablet_service.cc:1467] Processing CreateTablet for tablet 2c45107d61434583a568573b7a2d3e4c (DEFAULT_TABLE table=test-table [id=c4f4b99742744e3ab40e269d94fc0e4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:31:21.365692 29174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c45107d61434583a568573b7a2d3e4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:31:21.369923 29535 tablet_bootstrap.cc:654] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7: Neither blocks nor log segments found. Creating new log.
I20260619 02:31:21.355063 29450 tablet_service.cc:1467] Processing CreateTablet for tablet 2c45107d61434583a568573b7a2d3e4c (DEFAULT_TABLE table=test-table [id=c4f4b99742744e3ab40e269d94fc0e4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:31:21.371299 29450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c45107d61434583a568573b7a2d3e4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:31:21.371470 29535 log.cc:824] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7: Log is configured to *not* fsync() on all Append() calls
I20260619 02:31:21.373939 29535 tablet_bootstrap.cc:492] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7: No bootstrap required, opened a new log
I20260619 02:31:21.374204 29535 ts_tablet_manager.cc:1397] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.003s
I20260619 02:31:21.378835 29535 raft_consensus.cc:348] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7b2eb3ee64456499fac79e31e873b7" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 45663 } } peers { permanent_uuid: "6f7ef65063e84640a81ea1d29e2593c2" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 33007 } } peers { permanent_uuid: "3d3155bdbc0e4e63ae8241763f65c14e" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 39417 } }
I20260619 02:31:21.379180 29535 raft_consensus.cc:374] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:31:21.379244 29535 raft_consensus.cc:729] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c7b2eb3ee64456499fac79e31e873b7, State: Initialized, Role: FOLLOWER
I20260619 02:31:21.379786 29535 consensus_queue.cc:260] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7b2eb3ee64456499fac79e31e873b7" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 45663 } } peers { permanent_uuid: "6f7ef65063e84640a81ea1d29e2593c2" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 33007 } } peers { permanent_uuid: "3d3155bdbc0e4e63ae8241763f65c14e" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 39417 } }
I20260619 02:31:21.381014 29535 ts_tablet_manager.cc:1428] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260619 02:31:21.381428 29383 heartbeater.cc:503] Master 127.17.96.254:42427 was elected leader, sending a full tablet report...
I20260619 02:31:21.390185 29538 tablet_bootstrap.cc:492] T 2c45107d61434583a568573b7a2d3e4c P 6f7ef65063e84640a81ea1d29e2593c2: Bootstrap starting.
I20260619 02:31:21.394374 29538 tablet_bootstrap.cc:654] T 2c45107d61434583a568573b7a2d3e4c P 6f7ef65063e84640a81ea1d29e2593c2: Neither blocks nor log segments found. Creating new log.
I20260619 02:31:21.394968 29539 tablet_bootstrap.cc:492] T 2c45107d61434583a568573b7a2d3e4c P 3d3155bdbc0e4e63ae8241763f65c14e: Bootstrap starting.
I20260619 02:31:21.395468 29538 log.cc:824] T 2c45107d61434583a568573b7a2d3e4c P 6f7ef65063e84640a81ea1d29e2593c2: Log is configured to *not* fsync() on all Append() calls
I20260619 02:31:21.397873 29539 tablet_bootstrap.cc:654] T 2c45107d61434583a568573b7a2d3e4c P 3d3155bdbc0e4e63ae8241763f65c14e: Neither blocks nor log segments found. Creating new log.
I20260619 02:31:21.398298 29538 tablet_bootstrap.cc:492] T 2c45107d61434583a568573b7a2d3e4c P 6f7ef65063e84640a81ea1d29e2593c2: No bootstrap required, opened a new log
I20260619 02:31:21.398576 29538 ts_tablet_manager.cc:1397] T 2c45107d61434583a568573b7a2d3e4c P 6f7ef65063e84640a81ea1d29e2593c2: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260619 02:31:21.400569 29539 log.cc:824] T 2c45107d61434583a568573b7a2d3e4c P 3d3155bdbc0e4e63ae8241763f65c14e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:31:21.403390 29539 tablet_bootstrap.cc:492] T 2c45107d61434583a568573b7a2d3e4c P 3d3155bdbc0e4e63ae8241763f65c14e: No bootstrap required, opened a new log
I20260619 02:31:21.403262 29538 raft_consensus.cc:348] T 2c45107d61434583a568573b7a2d3e4c P 6f7ef65063e84640a81ea1d29e2593c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7b2eb3ee64456499fac79e31e873b7" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 45663 } } peers { permanent_uuid: "6f7ef65063e84640a81ea1d29e2593c2" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 33007 } } peers { permanent_uuid: "3d3155bdbc0e4e63ae8241763f65c14e" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 39417 } }
I20260619 02:31:21.403602 29538 raft_consensus.cc:374] T 2c45107d61434583a568573b7a2d3e4c P 6f7ef65063e84640a81ea1d29e2593c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:31:21.403648 29539 ts_tablet_manager.cc:1397] T 2c45107d61434583a568573b7a2d3e4c P 3d3155bdbc0e4e63ae8241763f65c14e: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260619 02:31:21.403656 29538 raft_consensus.cc:729] T 2c45107d61434583a568573b7a2d3e4c P 6f7ef65063e84640a81ea1d29e2593c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f7ef65063e84640a81ea1d29e2593c2, State: Initialized, Role: FOLLOWER
I20260619 02:31:21.404237 29538 consensus_queue.cc:260] T 2c45107d61434583a568573b7a2d3e4c P 6f7ef65063e84640a81ea1d29e2593c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7b2eb3ee64456499fac79e31e873b7" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 45663 } } peers { permanent_uuid: "6f7ef65063e84640a81ea1d29e2593c2" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 33007 } } peers { permanent_uuid: "3d3155bdbc0e4e63ae8241763f65c14e" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 39417 } }
I20260619 02:31:21.405483 29538 ts_tablet_manager.cc:1428] T 2c45107d61434583a568573b7a2d3e4c P 6f7ef65063e84640a81ea1d29e2593c2: Time spent starting tablet: real 0.007s user 0.000s sys 0.007s
I20260619 02:31:21.406011 29517 heartbeater.cc:503] Master 127.17.96.254:42427 was elected leader, sending a full tablet report...
I20260619 02:31:21.408912 29539 raft_consensus.cc:348] T 2c45107d61434583a568573b7a2d3e4c P 3d3155bdbc0e4e63ae8241763f65c14e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7b2eb3ee64456499fac79e31e873b7" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 45663 } } peers { permanent_uuid: "6f7ef65063e84640a81ea1d29e2593c2" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 33007 } } peers { permanent_uuid: "3d3155bdbc0e4e63ae8241763f65c14e" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 39417 } }
I20260619 02:31:21.409390 29539 raft_consensus.cc:374] T 2c45107d61434583a568573b7a2d3e4c P 3d3155bdbc0e4e63ae8241763f65c14e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:31:21.409507 29539 raft_consensus.cc:729] T 2c45107d61434583a568573b7a2d3e4c P 3d3155bdbc0e4e63ae8241763f65c14e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d3155bdbc0e4e63ae8241763f65c14e, State: Initialized, Role: FOLLOWER
I20260619 02:31:21.410195 29539 consensus_queue.cc:260] T 2c45107d61434583a568573b7a2d3e4c P 3d3155bdbc0e4e63ae8241763f65c14e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7b2eb3ee64456499fac79e31e873b7" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 45663 } } peers { permanent_uuid: "6f7ef65063e84640a81ea1d29e2593c2" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 33007 } } peers { permanent_uuid: "3d3155bdbc0e4e63ae8241763f65c14e" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 39417 } }
I20260619 02:31:21.412176 29247 heartbeater.cc:503] Master 127.17.96.254:42427 was elected leader, sending a full tablet report...
I20260619 02:31:21.413287 29539 ts_tablet_manager.cc:1428] T 2c45107d61434583a568573b7a2d3e4c P 3d3155bdbc0e4e63ae8241763f65c14e: Time spent starting tablet: real 0.010s user 0.004s sys 0.004s
Jun 19 02:31:21 dist-test-slave-4qw6 krb5kdc[28557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836280, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for HTTP/127.17.96.212@KRBTEST.COM
W20260619 02:31:21.464200 29248 tablet.cc:2369] T 2c45107d61434583a568573b7a2d3e4c P 3d3155bdbc0e4e63ae8241763f65c14e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:31:21.465232 29537 raft_consensus.cc:482] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:31:21.465461 29537 raft_consensus.cc:504] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7b2eb3ee64456499fac79e31e873b7" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 45663 } } peers { permanent_uuid: "6f7ef65063e84640a81ea1d29e2593c2" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 33007 } } peers { permanent_uuid: "3d3155bdbc0e4e63ae8241763f65c14e" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 39417 } }
I20260619 02:31:21.467155 29537 leader_election.cc:288] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f7ef65063e84640a81ea1d29e2593c2 (127.17.96.195:33007), 3d3155bdbc0e4e63ae8241763f65c14e (127.17.96.193:39417)
I20260619 02:31:21.477154 29534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:31:21.468003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:39003 (local address 127.17.96.193:39417)
0619 02:31:21.468159 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 02:31:21.468166 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:31:21.469271 (+ 1105us) server_negotiation.cc:407] Connection header received
0619 02:31:21.469484 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:31:21.469488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:31:21.469539 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:31:21.469589 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:31:21.470285 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.471042 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.475664 (+ 4622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.476297 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.476407 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:31:21.476485 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:31:21.476880 (+ 395us) server_negotiation.cc:299] Negotiation successful
0619 02:31:21.476981 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 02:31:21.477985 29545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:31:21.467567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:39417 (local address 127.17.96.194:39003)
0619 02:31:21.469186 (+ 1619us) negotiation.cc:107] Waiting for socket to connect
0619 02:31:21.469203 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:31:21.469356 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:31:21.469977 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:31:21.469980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:31:21.470000 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:31:21.470142 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:31:21.470155 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.471211 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.471214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:31:21.472337 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:31:21.472349 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.476594 (+ 4245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.476598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:31:21.476704 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:31:21.476745 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 02:31:21.477657 (+ 912us) client_negotiation.cc:240] Negotiation successful
0619 02:31:21.477795 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1549,"thread_start_us":121,"threads_started":1}
I20260619 02:31:21.478950 29190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c45107d61434583a568573b7a2d3e4c" candidate_uuid: "7c7b2eb3ee64456499fac79e31e873b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d3155bdbc0e4e63ae8241763f65c14e" is_pre_election: true
I20260619 02:31:21.479346 29190 raft_consensus.cc:2454] T 2c45107d61434583a568573b7a2d3e4c P 3d3155bdbc0e4e63ae8241763f65c14e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c7b2eb3ee64456499fac79e31e873b7 in term 0.
I20260619 02:31:21.480332 29533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:31:21.468900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:55249 (local address 127.17.96.195:33007)
0619 02:31:21.469025 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 02:31:21.469030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:31:21.472726 (+ 3696us) server_negotiation.cc:407] Connection header received
0619 02:31:21.472918 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:31:21.472921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:31:21.472978 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:31:21.473028 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:31:21.473483 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.474193 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.478736 (+ 4543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.479378 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.479478 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:31:21.479530 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:31:21.480093 (+ 563us) server_negotiation.cc:299] Negotiation successful
0619 02:31:21.480196 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 02:31:21.480963 29271 leader_election.cc:302] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d3155bdbc0e4e63ae8241763f65c14e, 7c7b2eb3ee64456499fac79e31e873b7; no voters:
I20260619 02:31:21.481528 29537 raft_consensus.cc:2781] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:31:21.481626 29537 raft_consensus.cc:482] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:31:21.481669 29537 raft_consensus.cc:3037] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:31:21.483249 29537 raft_consensus.cc:504] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7b2eb3ee64456499fac79e31e873b7" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 45663 } } peers { permanent_uuid: "6f7ef65063e84640a81ea1d29e2593c2" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 33007 } } peers { permanent_uuid: "3d3155bdbc0e4e63ae8241763f65c14e" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 39417 } }
I20260619 02:31:21.483677 29537 leader_election.cc:288] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7 [CANDIDATE]: Term 1 election: Requested vote from peers 6f7ef65063e84640a81ea1d29e2593c2 (127.17.96.195:33007), 3d3155bdbc0e4e63ae8241763f65c14e (127.17.96.193:39417)
I20260619 02:31:21.484441 29190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c45107d61434583a568573b7a2d3e4c" candidate_uuid: "7c7b2eb3ee64456499fac79e31e873b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d3155bdbc0e4e63ae8241763f65c14e"
I20260619 02:31:21.484593 29190 raft_consensus.cc:3037] T 2c45107d61434583a568573b7a2d3e4c P 3d3155bdbc0e4e63ae8241763f65c14e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:31:21.486117 29190 raft_consensus.cc:2454] T 2c45107d61434583a568573b7a2d3e4c P 3d3155bdbc0e4e63ae8241763f65c14e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c7b2eb3ee64456499fac79e31e873b7 in term 1.
I20260619 02:31:21.486604 29271 leader_election.cc:302] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d3155bdbc0e4e63ae8241763f65c14e, 7c7b2eb3ee64456499fac79e31e873b7; no voters:
I20260619 02:31:21.486871 29537 raft_consensus.cc:2781] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:31:21.487160 29537 raft_consensus.cc:686] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7 [term 1 LEADER]: Becoming Leader. State: Replica: 7c7b2eb3ee64456499fac79e31e873b7, State: Running, Role: LEADER
I20260619 02:31:21.487565 29537 consensus_queue.cc:237] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7b2eb3ee64456499fac79e31e873b7" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 45663 } } peers { permanent_uuid: "6f7ef65063e84640a81ea1d29e2593c2" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 33007 } } peers { permanent_uuid: "3d3155bdbc0e4e63ae8241763f65c14e" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 39417 } }
I20260619 02:31:21.491425 29033 catalog_manager.cc:5495] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c7b2eb3ee64456499fac79e31e873b7 (127.17.96.194). New cstate: current_term: 1 leader_uuid: "7c7b2eb3ee64456499fac79e31e873b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c7b2eb3ee64456499fac79e31e873b7" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 45663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f7ef65063e84640a81ea1d29e2593c2" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 33007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d3155bdbc0e4e63ae8241763f65c14e" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 39417 } health_report { overall_health: UNKNOWN } } }
I20260619 02:31:21.495198 29546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:31:21.468484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:33007 (local address 127.17.96.194:55249)
0619 02:31:21.472624 (+ 4140us) negotiation.cc:107] Waiting for socket to connect
0619 02:31:21.472643 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:31:21.472801 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:31:21.473191 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:31:21.473194 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:31:21.473212 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:31:21.473348 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:31:21.473355 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.474376 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.474382 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:31:21.475494 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:31:21.475506 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.479775 (+ 4269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.479780 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:31:21.479907 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:31:21.479934 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 02:31:21.493358 (+ 13424us) client_negotiation.cc:240] Negotiation successful
0619 02:31:21.493492 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4061,"thread_start_us":117,"threads_started":1}
I20260619 02:31:21.496333 29471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c45107d61434583a568573b7a2d3e4c" candidate_uuid: "7c7b2eb3ee64456499fac79e31e873b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f7ef65063e84640a81ea1d29e2593c2"
I20260619 02:31:21.496369 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c45107d61434583a568573b7a2d3e4c" candidate_uuid: "7c7b2eb3ee64456499fac79e31e873b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f7ef65063e84640a81ea1d29e2593c2" is_pre_election: true
I20260619 02:31:21.496699 29452 raft_consensus.cc:2454] T 2c45107d61434583a568573b7a2d3e4c P 6f7ef65063e84640a81ea1d29e2593c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c7b2eb3ee64456499fac79e31e873b7 in term 0.
I20260619 02:31:21.515451 29530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:31:21.509197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42942 (local address 127.17.96.194:45663)
0619 02:31:21.509321 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0619 02:31:21.509327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:31:21.509349 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:31:21.509445 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:31:21.509448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:31:21.509520 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:31:21.509653 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:31:21.510342 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.511330 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.512666 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:31:21.513010 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:31:21.513137 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:31:21.513956 (+ 819us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:31:21.514616 (+ 660us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:31:21.514755 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 02:31:21.515198 (+ 443us) server_negotiation.cc:299] Negotiation successful
0619 02:31:21.515280 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260619 02:31:21.527763 29518 tablet.cc:2369] T 2c45107d61434583a568573b7a2d3e4c P 6f7ef65063e84640a81ea1d29e2593c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:31:21.527875 29190 raft_consensus.cc:1261] T 2c45107d61434583a568573b7a2d3e4c P 3d3155bdbc0e4e63ae8241763f65c14e [term 1 FOLLOWER]: Refusing update from remote peer 7c7b2eb3ee64456499fac79e31e873b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:31:21.529155 29471 raft_consensus.cc:3037] T 2c45107d61434583a568573b7a2d3e4c P 6f7ef65063e84640a81ea1d29e2593c2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:31:21.530938 29471 raft_consensus.cc:1261] T 2c45107d61434583a568573b7a2d3e4c P 6f7ef65063e84640a81ea1d29e2593c2 [term 1 FOLLOWER]: Refusing update from remote peer 7c7b2eb3ee64456499fac79e31e873b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:31:21.531443 29547 consensus_queue.cc:1035] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d3155bdbc0e4e63ae8241763f65c14e" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 39417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:31:21.532058 29537 consensus_queue.cc:1035] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7ef65063e84640a81ea1d29e2593c2" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:31:21.561841 29551 mvcc.cc:205] Tried to move back new op lower bound from 7298401409121853440 to 7298401408977440768. Current Snapshot: MvccSnapshot[applied={T|T < 7298401409121853440 or (T in {7298401409121853440})}]
I20260619 02:31:21.564545 29033 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:31:21.564765 29033 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:31:21.567953 29033 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7cde35ffc93748a2ad65b6a483867cd2: Sending DeleteTablet for 3 replicas of tablet 2c45107d61434583a568573b7a2d3e4c
I20260619 02:31:21.569200 29303 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c45107d61434583a568573b7a2d3e4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:31:21 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:42932
I20260619 02:31:21.570587 29450 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c45107d61434583a568573b7a2d3e4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:31:21 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:36336
I20260619 02:31:21.570987 29174 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c45107d61434583a568573b7a2d3e4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:31:21 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:55120
I20260619 02:31:21.571867 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 29114
I20260619 02:31:21.575956 29553 mvcc.cc:205] Tried to move back new op lower bound from 7298401409121853440 to 7298401408977440768. Current Snapshot: MvccSnapshot[applied={T|T < 7298401409121853440}]
I20260619 02:31:21.586050 29559 tablet_replica.cc:330] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7: stopping tablet replica
W20260619 02:31:21.591789 29271 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.96.193:39417 (error 108)
I20260619 02:31:21.592873 29559 raft_consensus.cc:2229] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:31:21.593423 29559 raft_consensus.cc:2258] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:31:21.594053 29560 tablet_replica.cc:330] T 2c45107d61434583a568573b7a2d3e4c P 6f7ef65063e84640a81ea1d29e2593c2: stopping tablet replica
I20260619 02:31:21.594534 29560 raft_consensus.cc:2229] T 2c45107d61434583a568573b7a2d3e4c P 6f7ef65063e84640a81ea1d29e2593c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:31:21.594986 29560 raft_consensus.cc:2258] T 2c45107d61434583a568573b7a2d3e4c P 6f7ef65063e84640a81ea1d29e2593c2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 02:31:21.595679 29018 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.96.193:39417 (error 108)
I20260619 02:31:21.595760 29559 ts_tablet_manager.cc:1905] T 2c45107d61434583a568573b7a2d3e4c P 7c7b2eb3ee64456499fac79e31e873b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:31:21.598053 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 29253
I20260619 02:31:21.603255 29560 ts_tablet_manager.cc:1905] T 2c45107d61434583a568573b7a2d3e4c P 6f7ef65063e84640a81ea1d29e2593c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:31:21.606710 29560 ts_tablet_manager.cc:1918] T 2c45107d61434583a568573b7a2d3e4c P 6f7ef65063e84640a81ea1d29e2593c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:31:21.607450 29560 log.cc:1196] T 2c45107d61434583a568573b7a2d3e4c P 6f7ef65063e84640a81ea1d29e2593c2: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/2c45107d61434583a568573b7a2d3e4c
I20260619 02:31:21.607571 29532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:31:21.607242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:39417 (local address 127.0.0.1:55130)
0619 02:31:21.607385 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 02:31:21.607474 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.193:39417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260619 02:31:21.607919 29018 catalog_manager.cc:4584] TS 3d3155bdbc0e4e63ae8241763f65c14e (127.17.96.193:39417): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2c45107d61434583a568573b7a2d3e4c: Network error: Client connection negotiation failed: client connection to 127.17.96.193:39417: connect: Connection refused (error 111)
I20260619 02:31:21.608381 29560 ts_tablet_manager.cc:1939] T 2c45107d61434583a568573b7a2d3e4c P 6f7ef65063e84640a81ea1d29e2593c2: Deleting consensus metadata
I20260619 02:31:21.610029 29020 catalog_manager.cc:4841] TS 6f7ef65063e84640a81ea1d29e2593c2 (127.17.96.195:33007): tablet 2c45107d61434583a568573b7a2d3e4c (table test-table [id=c4f4b99742744e3ab40e269d94fc0e4a]) successfully deleted
I20260619 02:31:21.615937 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 29388
I20260619 02:31:21.616348 29532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:31:21.615996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:45663 (local address 127.0.0.1:42958)
0619 02:31:21.616202 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 02:31:21.616258 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.194:45663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260619 02:31:21.623929 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 29001
I20260619 02:31:21.631068 17795 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 02:31:21.976845 17795 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 02:31:21.977030 17795 mini_ranger.cc:66] Stopping Ranger...
I20260619 02:31:22.322646 17795 mini_ranger.cc:68] Stopped Ranger
2026-06-19 02:31:22.322 UTC [28587] LOG: received smart shutdown request
2026-06-19 02:31:22.326 UTC [28587] LOG: background worker "logical replication launcher" (PID 28596) exited with exit code 1
2026-06-19 02:31:22.326 UTC [28591] LOG: shutting down
2026-06-19 02:31:22.497 UTC [28587] LOG: database system is shut down
2026-06-19T02:31:22Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (52563 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:31:22 dist-test-slave-4qw6 krb5kdc[29572](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:31:22 dist-test-slave-4qw6 krb5kdc[29572](info): set up 2 sockets
Jun 19 02:31:22 dist-test-slave-4qw6 krb5kdc[29572](info): commencing operation
krb5kdc: starting...
W20260619 02:31:23.665823 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:31:23 dist-test-slave-4qw6 krb5kdc[29572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:31:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:31:23Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.96.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.96.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.96.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/rangeradmin_127.17.96.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.96.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/rangeradmin_127.17.96.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.96.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.96.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.96.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/rangerlookup_127.17.96.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.96.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/rangerlookup_127.17.96.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.96.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.96.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.96.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/HTTP_127.17.96.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.96.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/HTTP_127.17.96.212@KRBTEST.COM.keytab.
I20260619 02:31:23.843899 17795 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskhpFBxW/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/postgres -l logfile start
2026-06-19 02:31:26.376 UTC [29602] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 02:31:26.377 UTC [29602] LOG: listening on IPv4 address "127.17.96.212", port 35173
2026-06-19 02:31:26.380 UTC [29602] LOG: listening on Unix socket "/tmp/.s.PGSQL.35173"
2026-06-19 02:31:26.402 UTC [29605] LOG: database system was shut down at 2026-06-19 02:31:24 UTC
2026-06-19 02:31:26.406 UTC [29602] LOG: database system is ready to accept connections
I20260619 02:31:27.357703 17795 mini_postgres.cc:96] Postgres bound to 35173
2026-06-19 02:31:27.364 UTC [29613] FATAL: database "slave" does not exist
127.17.96.212:35173 - accepting connections
I20260619 02:31:27.364825 17795 mini_ranger.cc:161] Starting Ranger...
I20260619 02:31:27.380546 17795 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 02:31:27.856765 17795 mini_ranger.cc:87] Created ranger Postgres database
I20260619 02:31:27.856848 17795 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ranger-admin
2026-06-19 02:31:28,150 [I] DB FLAVOR :POSTGRES
2026-06-19 02:31:28,151 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 02:31:28,151 [I] Checking connection..
2026-06-19 02:31:28,151 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:35173/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 02:31:28,418 [I] Checking connection passed.
2026-06-19 02:31:28,418 [I] --------- Verifying version history table ---------
2026-06-19 02:31:28,418 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:35173/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:31:28,671 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 02:31:28,671 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:35173/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:31:28,945 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 02:31:28,945 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 02:31:28,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:35173/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 02:31:29.185 UTC [29677] WARNING: there is no transaction in progress
2026-06-19 02:31:29,192 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 02:31:29,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:35173/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:31:29,470 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 02:31:29,470 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 02:31:29,470 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:35173/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 02:31:29,725 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:35173/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 02:31:29,990 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:35173/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-4qw6.c.gcp-upstream.internal','N') ;"
2026-06-19 02:31:30,231 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:35173/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 02:31:30,506 [I] Table x_portal_user does not exist in database ranger
2026-06-19 02:31:30,507 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:35173/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 02:31:30,788 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 02:31:30,789 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:35173/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 02:31:31,063 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:35173/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 02:31:31,318 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 02:31:31,318 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:35173/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 02:31:32.356 UTC [29785] WARNING: there is no transaction in progress
2026-06-19 02:31:32.373 UTC [29785] WARNING: there is no transaction in progress
2026-06-19 02:31:32.390 UTC [29785] WARNING: there is no transaction in progress
2026-06-19 02:31:32.406 UTC [29785] WARNING: there is no transaction in progress
2026-06-19 02:31:32.422 UTC [29785] WARNING: there is no transaction in progress
2026-06-19 02:31:32.442 UTC [29785] WARNING: there is no transaction in progress
2026-06-19 02:31:32.500 UTC [29785] WARNING: there is no transaction in progress
2026-06-19 02:31:32.507 UTC [29785] WARNING: there is no transaction in progress
2026-06-19 02:31:32.523 UTC [29785] WARNING: there is no transaction in progress
2026-06-19 02:31:32.534 UTC [29785] WARNING: there is no transaction in progress
2026-06-19 02:31:32.544 UTC [29785] WARNING: there is no transaction in progress
2026-06-19 02:31:32.560 UTC [29785] WARNING: there is no transaction in progress
2026-06-19 02:31:32.571 UTC [29785] WARNING: there is no transaction in progress
2026-06-19 02:31:32.578 UTC [29785] WARNING: there is no transaction in progress
2026-06-19 02:31:32.586 UTC [29785] WARNING: there is no transaction in progress
2026-06-19 02:31:32.592 UTC [29785] WARNING: there is no transaction in progress
2026-06-19 02:31:33,279 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 02:31:33,280 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:35173/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 02:31:33,513 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 02:31:33,513 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:35173/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 02:31:33,788 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 02:31:33,789 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:35173/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 02:31:34,067 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 02:31:34,067 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:35173/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 02:31:34,329 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:35173/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 02:31:34,577 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:35173/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-4qw6.c.gcp-upstream.internal';"
2026-06-19 02:31:34,857 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 02:31:34,858 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:35173/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 02:31:35,135 [I] DB_PATCHES have already been applied
I20260619 02:31:35.141213 17795 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ranger-admin:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskhpFBxW/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 02:31:35.141330 17795 mini_ranger.cc:193] Using host: 127.17.96.212
I20260619 02:31:35.145083 17795 mini_ranger.cc:227] Ranger admin URL: http://127.17.96.212:42629
Jun 19, 2026 2:31:35 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:31:35 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:31:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 2:31:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 2:31:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 2:31:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 2:31:36 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 02:31:36.302757 17795 mini_ranger.cc:160] Time spent starting Ranger: real 8.938s user 0.000s sys 0.011s
Jun 19, 2026 2:31:56 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 2:31:56 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 2:31:56 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 2:31:56 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 2:31:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 2:31:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 2:31:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 2:31:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 2:31:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 2:31:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 2:31:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 2:31:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 2:31:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 2:31:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 2:31:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 2:31:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 2:31:56 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 2:31:57 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 02:31:57.864578 17795 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.96.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.96.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.96.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/rangerkms_127.17.96.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.96.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/rangerkms_127.17.96.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.96.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.96.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.96.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/HTTP_127.17.96.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.96.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/HTTP_127.17.96.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 02:31:57 dist-test-slave-4qw6 krb5kdc[29572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836317, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 02:31:57.954888 17795 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 02:31:57.964975 17795 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 02:31:58.815212 17795 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 02:31:58.815311 17795 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ranger-kms
I20260619 02:31:58.815331 17795 mini_ranger_kms.cc:227] Using postgres at 127.17.96.212:35173
2026-06-19 02:31:58,990 [I] DB FLAVOR :POSTGRES
2026-06-19 02:31:58,990 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 02:31:58,990 [I] Checking connection
2026-06-19 02:31:58,990 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:35173/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 02:31:59,233 [I] connection success
2026-06-19 02:31:59,234 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 02:31:59,234 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 02:31:59,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:35173/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 02:31:59,476 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 02:31:59,476 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 02:31:59,476 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 02:31:59,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.96.212:35173/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 02:31:59,748 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 02:32:00.255141 17795 mini_ranger_kms.cc:310] Created kms service
I20260619 02:32:00.461246 17795 mini_ranger_kms.cc:326] Created kudu user
I20260619 02:32:00.508673 17795 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 02:32:00 dist-test-slave-4qw6 krb5kdc[29572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836317, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.96.212@KRBTEST.COM
I20260619 02:32:00.989523 17795 mini_ranger_kms.cc:384] Added ranger policy
I20260619 02:32:00.989709 17795 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ranger-kms:/tmp/dist-test-taskhpFBxW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskhpFBxW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 02:32:00.989730 17795 mini_ranger_kms.cc:240] Using host: 127.17.96.212
I20260619 02:32:00.994412 17795 mini_ranger_kms.cc:276] Ranger KMS PID: 29965
I20260619 02:32:00.994522 17795 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.96.212:42257
Jun 19, 2026 2:32:02 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:32:02 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:32:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 2:32:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 2:32:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 2:32:02 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 02:32:03.329560 17795 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.375s user 0.009s sys 0.149s
I20260619 02:32:03.329700 17795 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 02:32:03.329808 17795 mini_ranger_kms.cc:401] 127.17.96.212:42257/kms/v1/keys
Jun 19 02:32:03 dist-test-slave-4qw6 krb5kdc[29572](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781836323, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.96.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 02:32:03 dist-test-slave-4qw6 krb5kdc[29572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836323, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.96.212@KRBTEST.COM for HTTP/127.17.96.212@KRBTEST.COM
Jun 19 02:32:04 dist-test-slave-4qw6 krb5kdc[29572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836323, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.96.212@KRBTEST.COM for HTTP/127.17.96.212@KRBTEST.COM
Jun 19 02:32:06 dist-test-slave-4qw6 krb5kdc[29572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:32:06.660617 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:34729
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:36967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:34729
--ranger_config_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.96.212:42257/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:32:06.778663 30016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:06.779008 30016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:06.779083 30016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:06.783382 30016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:32:06.783495 30016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:06.783550 30016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:06.783608 30016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:32:06.783645 30016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:32:06.789846 30016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:36967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.96.212:42257/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:34729
--ranger_config_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:34729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:06.791092 30016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:06.792192 30016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:06.801465 30021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:06.801626 30024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:06.801465 30022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:06.801515 30016 server_base.cc:1034] running on GCE node
I20260619 02:32:06.802399 30016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:06.803608 30016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:06.804822 30016 hybrid_clock.cc:648] HybridClock initialized: now 1781836326804796 us; error 54 us; skew 500 ppm
Jun 19 02:32:06 dist-test-slave-4qw6 krb5kdc[29572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836326, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:06.808120 30016 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:32:06.809186 30016 webserver.cc:460] Webserver started at http://127.17.96.254:40535/ using document root <none> and password file <none>
I20260619 02:32:06.809900 30016 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:06.809993 30016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:06.810232 30016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:32:06 dist-test-slave-4qw6 krb5kdc[29572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836326, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for HTTP/127.17.96.212@KRBTEST.COM
I20260619 02:32:06.861673 30016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "1cc8a20fa02f4b198e68971c2b23c2da"
format_stamp: "Formatted at 2026-06-19 02:32:06 on dist-test-slave-4qw6"
server_key: "411c510efc1e1322531941f3235fb06a"
server_key_iv: "13f54fcecd7826d75bb99b90dcb8e66d"
server_key_version: "kuduclusterkey@0"
I20260619 02:32:06.862430 30016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "1cc8a20fa02f4b198e68971c2b23c2da"
format_stamp: "Formatted at 2026-06-19 02:32:06 on dist-test-slave-4qw6"
server_key: "411c510efc1e1322531941f3235fb06a"
server_key_iv: "13f54fcecd7826d75bb99b90dcb8e66d"
server_key_version: "kuduclusterkey@0"
I20260619 02:32:06.866709 30016 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 02:32:06.885653 30031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:06.887184 30016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:32:06.887312 30016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "1cc8a20fa02f4b198e68971c2b23c2da"
format_stamp: "Formatted at 2026-06-19 02:32:06 on dist-test-slave-4qw6"
server_key: "411c510efc1e1322531941f3235fb06a"
server_key_iv: "13f54fcecd7826d75bb99b90dcb8e66d"
server_key_version: "kuduclusterkey@0"
I20260619 02:32:06.887486 30016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:06.903311 30016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:06.906488 30016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:06.906699 30016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:06.915125 30016 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:34729
I20260619 02:32:06.915127 30083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:34729 every 8 connection(s)
I20260619 02:32:06.916225 30016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:32:06.919341 30084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:32:06.919447 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 30016
I20260619 02:32:06.919574 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:32:06.919866 17795 external_mini_cluster.cc:1428] Setting key 6b367b24d634390879336bd909759a40
I20260619 02:32:06.927150 30084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da: Bootstrap starting.
I20260619 02:32:06.930079 30084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da: Neither blocks nor log segments found. Creating new log.
Jun 19 02:32:06 dist-test-slave-4qw6 krb5kdc[29572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:06.931139 30084 log.cc:824] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da: Log is configured to *not* fsync() on all Append() calls
I20260619 02:32:06.933564 30084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da: No bootstrap required, opened a new log
I20260619 02:32:06.937422 30084 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc8a20fa02f4b198e68971c2b23c2da" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34729 } }
I20260619 02:32:06.937698 30084 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:32:06.937753 30084 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc8a20fa02f4b198e68971c2b23c2da, State: Initialized, Role: FOLLOWER
I20260619 02:32:06.938301 30087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:06.921526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59894 (local address 127.17.96.254:34729)
0619 02:32:06.922039 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:06.922060 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:06.922106 (+ 46us) server_negotiation.cc:407] Connection header received
0619 02:32:06.922991 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:06.923016 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:06.923492 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:06.923886 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:06.924857 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:06.926404 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:06.927655 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:06.928346 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:06.928544 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:06.931749 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:06.931755 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:06.931772 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:06.931802 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:06.934679 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:06.935295 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:06.935302 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:06.935308 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:06.935456 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:06.935793 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:06.935797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:06.935798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:06.936344 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:06.936537 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:06.936883 (+ 346us) server_negotiation.cc:299] Negotiation successful
0619 02:32:06.937203 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":166,"threads_started":1}
I20260619 02:32:06.938262 30084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc8a20fa02f4b198e68971c2b23c2da" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34729 } }
I20260619 02:32:06.938491 30084 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:32:06.938567 30084 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:32:06.938684 30084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:32:06.939934 30084 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc8a20fa02f4b198e68971c2b23c2da" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34729 } }
I20260619 02:32:06.940304 30084 leader_election.cc:302] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cc8a20fa02f4b198e68971c2b23c2da; no voters:
I20260619 02:32:06.940706 30084 leader_election.cc:288] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:32:06.941185 30089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:32:06.941434 30089 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [term 1 LEADER]: Becoming Leader. State: Replica: 1cc8a20fa02f4b198e68971c2b23c2da, State: Running, Role: LEADER
I20260619 02:32:06.941830 30089 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc8a20fa02f4b198e68971c2b23c2da" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34729 } }
I20260619 02:32:06.942090 30084 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:32:06.943542 30091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cc8a20fa02f4b198e68971c2b23c2da. Latest consensus state: current_term: 1 leader_uuid: "1cc8a20fa02f4b198e68971c2b23c2da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc8a20fa02f4b198e68971c2b23c2da" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34729 } } }
I20260619 02:32:06.943670 30091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [sys.catalog]: This master's current role is: LEADER
I20260619 02:32:06.943981 30090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1cc8a20fa02f4b198e68971c2b23c2da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc8a20fa02f4b198e68971c2b23c2da" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34729 } } }
I20260619 02:32:06.944048 30090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [sys.catalog]: This master's current role is: LEADER
I20260619 02:32:06.944398 30084 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 02:32:06.944511 30100 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:32:06.951144 30100 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:32:06.961220 30100 catalog_manager.cc:1347] Generated new cluster ID: 7a085fee84814aed80e9b274a0cae196
I20260619 02:32:06.961313 30100 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:32:06.982381 30100 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:32:06.984634 30100 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:32:06.993201 30100 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da: Generated new TSK 0
I20260619 02:32:06.994222 30100 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.96.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
I20260619 02:32:07.742456 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.96.212:42257/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:34729
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:36967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:32:07.917470 30129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:07.917979 30129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:07.918105 30129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:07.925132 30129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:07.925319 30129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:07.925469 30129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:32:07.934252 30129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:36967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.96.212:42257/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:34729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:07.935983 30129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:07.937597 30129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:07.951179 30138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:07.951718 30136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:07.951781 30129 server_base.cc:1034] running on GCE node
W20260619 02:32:07.951959 30135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:07.952409 30129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:07.953166 30129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:07.954362 30129 hybrid_clock.cc:648] HybridClock initialized: now 1781836327954323 us; error 47 us; skew 500 ppm
Jun 19 02:32:07 dist-test-slave-4qw6 krb5kdc[29572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836327, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:07.958284 30129 init.cc:377] Logged in from keytab as kudu/127.17.96.193@KRBTEST.COM (short username kudu)
I20260619 02:32:07.959585 30129 webserver.cc:460] Webserver started at http://127.17.96.193:37555/ using document root <none> and password file <none>
I20260619 02:32:07.960218 30129 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:07.960326 30129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:07.960561 30129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:32:07 dist-test-slave-4qw6 krb5kdc[29572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836327, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for HTTP/127.17.96.212@KRBTEST.COM
I20260619 02:32:07.980978 30129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "ce896a6be62b4a9d9df161920546a682"
format_stamp: "Formatted at 2026-06-19 02:32:07 on dist-test-slave-4qw6"
server_key: "fc2b3078c3cc78949825c758efac3e52"
server_key_iv: "1539d826b1ad46f7eb0fc817807df9be"
server_key_version: "kuduclusterkey@0"
I20260619 02:32:07.981971 30129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "ce896a6be62b4a9d9df161920546a682"
format_stamp: "Formatted at 2026-06-19 02:32:07 on dist-test-slave-4qw6"
server_key: "fc2b3078c3cc78949825c758efac3e52"
server_key_iv: "1539d826b1ad46f7eb0fc817807df9be"
server_key_version: "kuduclusterkey@0"
I20260619 02:32:07.991657 30129 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.000s
I20260619 02:32:08.011595 30146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:08.013388 30129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:32:08.013542 30129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "ce896a6be62b4a9d9df161920546a682"
format_stamp: "Formatted at 2026-06-19 02:32:07 on dist-test-slave-4qw6"
server_key: "fc2b3078c3cc78949825c758efac3e52"
server_key_iv: "1539d826b1ad46f7eb0fc817807df9be"
server_key_version: "kuduclusterkey@0"
I20260619 02:32:08.013686 30129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:08.044062 30129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:08.048516 30129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:08.048750 30129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:08.049579 30129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:32:08.050715 30129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:32:08.050789 30129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:08.050858 30129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:32:08.050890 30129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:08.062466 30129 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:36749
I20260619 02:32:08.063797 30129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:32:08.065739 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 30129
I20260619 02:32:08.065910 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:32:08.066273 17795 external_mini_cluster.cc:1428] Setting key d6011a52e9e652beb20fed72c5861478
I20260619 02:32:08.076947 30259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:36749 every 8 connection(s)
WARNING: no policy specified for kudu/127.17.96.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.194@KRBTEST.COM" created.
Jun 19 02:32:08 dist-test-slave-4qw6 krb5kdc[29572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836327, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:08.102285 30265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:08.080334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:34729 (local address 127.17.96.193:52663)
0619 02:32:08.082791 (+ 2457us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:08.082899 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:08.084067 (+ 1168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:08.085158 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:08.085169 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:08.085602 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:08.086466 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:08.086486 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:08.090082 (+ 3596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:08.090086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:08.091709 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:08.091724 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:08.092630 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:08.092634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:08.092859 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:08.093696 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:08.093720 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:08.095678 (+ 1958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:08.098252 (+ 2574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:08.098260 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:08.098281 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:08.098700 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:08.099103 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:08.099107 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:08.099109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:08.099338 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:08.099882 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:08.099891 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:08.100319 (+ 428us) client_negotiation.cc:769] Sending connection context
0619 02:32:08.100565 (+ 246us) client_negotiation.cc:240] Negotiation successful
0619 02:32:08.100989 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2187,"thread_start_us":157,"threads_started":1}
I20260619 02:32:08.103426 30266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:08.081537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:52663 (local address 127.17.96.254:34729)
0619 02:32:08.084581 (+ 3044us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:08.084586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:08.084602 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:32:08.084648 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:08.084650 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:08.084706 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:08.084796 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:08.088882 (+ 4086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:08.089857 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:08.091889 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:08.092299 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:08.092414 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:08.095933 (+ 3519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:08.095938 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:08.095941 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:08.095974 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:08.098061 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:08.098851 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:08.098856 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:08.098858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:08.098968 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:08.099462 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:08.099465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:08.099467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:08.099721 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:08.099820 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:08.103020 (+ 3200us) server_negotiation.cc:299] Negotiation successful
0619 02:32:08.103220 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2949,"thread_start_us":60,"threads_started":1}
I20260619 02:32:08.109171 30260 heartbeater.cc:347] Connected to a master server at 127.17.96.254:34729
I20260619 02:32:08.109474 30260 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:08.110397 30260 heartbeater.cc:511] Master 127.17.96.254:34729 requested a full tablet report, sending...
I20260619 02:32:08.112524 30048 ts_manager.cc:194] Registered new tserver with Master: ce896a6be62b4a9d9df161920546a682 (127.17.96.193:36749)
I20260619 02:32:08.114270 30048 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:52663
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
I20260619 02:32:08.133908 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.96.212:42257/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:34729
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:36967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:32:08 dist-test-slave-4qw6 krb5kdc[29572](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781836328, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 02:32:08.310173 30268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:08.310680 30268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:08.310765 30268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:08.317922 30268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:08.318043 30268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:08.318154 30268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:32:08.327631 30268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:36967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.96.212:42257/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:34729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:08.329391 30268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:08.330978 30268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:08.344485 30277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:08.344834 30275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:08.345290 30274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:08.346520 30268 server_base.cc:1034] running on GCE node
I20260619 02:32:08.347143 30268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:08.348135 30268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:08.349399 30268 hybrid_clock.cc:648] HybridClock initialized: now 1781836328349358 us; error 67 us; skew 500 ppm
Jun 19 02:32:08 dist-test-slave-4qw6 krb5kdc[29572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836328, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:08.352780 30268 init.cc:377] Logged in from keytab as kudu/127.17.96.194@KRBTEST.COM (short username kudu)
I20260619 02:32:08.353977 30268 webserver.cc:460] Webserver started at http://127.17.96.194:40923/ using document root <none> and password file <none>
I20260619 02:32:08.354632 30268 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:08.354734 30268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:08.354959 30268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:32:08 dist-test-slave-4qw6 krb5kdc[29572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836328, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for HTTP/127.17.96.212@KRBTEST.COM
I20260619 02:32:08.376493 30268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "861fd6bbc07b4da6bf07f2848d5d398d"
format_stamp: "Formatted at 2026-06-19 02:32:08 on dist-test-slave-4qw6"
server_key: "1161388b66fee8a1d72520aea8932c42"
server_key_iv: "a857eb5cd502623fb6dcd5cf4a736942"
server_key_version: "kuduclusterkey@0"
I20260619 02:32:08.377816 30268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "861fd6bbc07b4da6bf07f2848d5d398d"
format_stamp: "Formatted at 2026-06-19 02:32:08 on dist-test-slave-4qw6"
server_key: "1161388b66fee8a1d72520aea8932c42"
server_key_iv: "a857eb5cd502623fb6dcd5cf4a736942"
server_key_version: "kuduclusterkey@0"
I20260619 02:32:08.387565 30268 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.000s
I20260619 02:32:08.415102 30284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:08.417062 30268 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260619 02:32:08.417223 30268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "861fd6bbc07b4da6bf07f2848d5d398d"
format_stamp: "Formatted at 2026-06-19 02:32:08 on dist-test-slave-4qw6"
server_key: "1161388b66fee8a1d72520aea8932c42"
server_key_iv: "a857eb5cd502623fb6dcd5cf4a736942"
server_key_version: "kuduclusterkey@0"
I20260619 02:32:08.417367 30268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:08.440608 30268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:08.444766 30268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:08.444975 30268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:08.445709 30268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:32:08.446853 30268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:32:08.446923 30268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:08.446986 30268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:32:08.447018 30268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:08.457897 30268 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:33721
I20260619 02:32:08.459148 30268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:32:08.460732 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 30268
I20260619 02:32:08.460863 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:32:08.461220 17795 external_mini_cluster.cc:1428] Setting key 3b4b12a14cd4c28bfd0f0a8482b90668
I20260619 02:32:08.465662 30397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:33721 every 8 connection(s)
WARNING: no policy specified for kudu/127.17.96.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.195@KRBTEST.COM" created.
Jun 19 02:32:08 dist-test-slave-4qw6 krb5kdc[29572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836328, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:08.484820 30266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:08.469508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:56351 (local address 127.17.96.254:34729)
0619 02:32:08.469641 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:08.469646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:08.469666 (+ 20us) server_negotiation.cc:407] Connection header received
0619 02:32:08.470753 (+ 1087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:08.470757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:08.470822 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:08.470880 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:08.472691 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:08.473565 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:08.475454 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:08.475848 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:08.475933 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:08.480703 (+ 4770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:08.480711 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:08.480718 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:08.480759 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:08.482431 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:08.483065 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:08.483070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:08.483072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:08.483166 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:08.483578 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:08.483581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:08.483583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:08.483794 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:08.483910 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:08.484494 (+ 584us) server_negotiation.cc:299] Negotiation successful
0619 02:32:08.484645 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 02:32:08.485947 30401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:08.468199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:34729 (local address 127.17.96.194:56351)
0619 02:32:08.469116 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:08.469185 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:08.470396 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:08.471339 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:08.471350 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:08.471760 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:08.472463 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:08.472482 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:08.473787 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:08.473791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:08.475286 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:08.475299 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:08.476173 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:08.476177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:08.476400 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:08.477143 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:08.477165 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:08.480419 (+ 3254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:08.482602 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:08.482609 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:08.482633 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:08.482937 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:08.483281 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:08.483284 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:08.483286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:08.483470 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:08.483910 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:08.483921 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:08.484229 (+ 308us) client_negotiation.cc:769] Sending connection context
0619 02:32:08.484444 (+ 215us) client_negotiation.cc:240] Negotiation successful
0619 02:32:08.484825 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":183,"threads_started":1}
I20260619 02:32:08.487352 30398 heartbeater.cc:347] Connected to a master server at 127.17.96.254:34729
I20260619 02:32:08.487633 30398 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:08.488276 30398 heartbeater.cc:511] Master 127.17.96.254:34729 requested a full tablet report, sending...
I20260619 02:32:08.489427 30048 ts_manager.cc:194] Registered new tserver with Master: 861fd6bbc07b4da6bf07f2848d5d398d (127.17.96.194:33721)
I20260619 02:32:08.490108 30048 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:56351
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
I20260619 02:32:08.502935 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.96.212:42257/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:34729
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:36967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:32:08.666862 30403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:08.667378 30403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:08.667506 30403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:08.674142 30403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:08.674335 30403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:08.674496 30403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:32:08.683156 30403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:36967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.96.212:42257/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:34729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
Jun 19 02:32:08 dist-test-slave-4qw6 krb5kdc[29572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836328, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for HTTP/127.17.96.212@KRBTEST.COM
I20260619 02:32:08.685820 30403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:08.687359 30403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:08.702145 30408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:08.704136 30411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:08.704457 30409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:08.711670 30403 server_base.cc:1034] running on GCE node
I20260619 02:32:08.712533 30403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:08.713543 30403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:08.716853 30403 hybrid_clock.cc:648] HybridClock initialized: now 1781836328716663 us; error 198 us; skew 500 ppm
Jun 19 02:32:08 dist-test-slave-4qw6 krb5kdc[29572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836328, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:08.722124 30403 init.cc:377] Logged in from keytab as kudu/127.17.96.195@KRBTEST.COM (short username kudu)
I20260619 02:32:08.723655 30403 webserver.cc:460] Webserver started at http://127.17.96.195:42817/ using document root <none> and password file <none>
I20260619 02:32:08.724351 30403 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:08.724481 30403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:08.724833 30403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:32:08 dist-test-slave-4qw6 krb5kdc[29572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836328, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for HTTP/127.17.96.212@KRBTEST.COM
I20260619 02:32:08.747679 30403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "79e08f569c3d42f0ab6abbcdf1636360"
format_stamp: "Formatted at 2026-06-19 02:32:08 on dist-test-slave-4qw6"
server_key: "18f8abba608a2bc0233b9d92e665473e"
server_key_iv: "7f5a54ff49b3fc9dc1b0657a4588e41a"
server_key_version: "kuduclusterkey@0"
I20260619 02:32:08.748709 30403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "79e08f569c3d42f0ab6abbcdf1636360"
format_stamp: "Formatted at 2026-06-19 02:32:08 on dist-test-slave-4qw6"
server_key: "18f8abba608a2bc0233b9d92e665473e"
server_key_iv: "7f5a54ff49b3fc9dc1b0657a4588e41a"
server_key_version: "kuduclusterkey@0"
I20260619 02:32:08.754716 30403 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 02:32:08.780637 30418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:08.782533 30403 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260619 02:32:08.782699 30403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "79e08f569c3d42f0ab6abbcdf1636360"
format_stamp: "Formatted at 2026-06-19 02:32:08 on dist-test-slave-4qw6"
server_key: "18f8abba608a2bc0233b9d92e665473e"
server_key_iv: "7f5a54ff49b3fc9dc1b0657a4588e41a"
server_key_version: "kuduclusterkey@0"
I20260619 02:32:08.782850 30403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:08.825805 30403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:08.830315 30403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:08.830642 30403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:08.831434 30403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:32:08.832706 30403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:32:08.832826 30403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:08.832938 30403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:32:08.833011 30403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:08.847854 30403 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:41983
I20260619 02:32:08.848009 30531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:41983 every 8 connection(s)
I20260619 02:32:08.848955 30403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:32:08.850234 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 30403
I20260619 02:32:08.850389 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:32:08.850720 17795 external_mini_cluster.cc:1428] Setting key 32d281904aa001ea0911b7b8cc4f6d14
Jun 19 02:32:08 dist-test-slave-4qw6 krb5kdc[29572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836328, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:08.872079 30266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:08.853692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:45203 (local address 127.17.96.254:34729)
0619 02:32:08.853882 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:08.853887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:08.858286 (+ 4399us) server_negotiation.cc:407] Connection header received
0619 02:32:08.859313 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:08.859317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:08.859377 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:08.859460 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:08.861205 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:08.862104 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:08.863903 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:08.864271 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:08.864348 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:08.867621 (+ 3273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:08.867625 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:08.867628 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:08.867659 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:08.869395 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:08.870167 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:08.870170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:08.870173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:08.870266 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:08.870718 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:08.870721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:08.870722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:08.870953 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:08.871047 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:08.871770 (+ 723us) server_negotiation.cc:299] Negotiation successful
0619 02:32:08.871909 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260619 02:32:08.873231 30534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:08.851948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:34729 (local address 127.17.96.195:45203)
0619 02:32:08.858092 (+ 6144us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:08.858150 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:08.859066 (+ 916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:08.859818 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:08.859829 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:08.860259 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:08.861010 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:08.861030 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:08.862277 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:08.862280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:08.863748 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:08.863760 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:08.864587 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:08.864591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:08.864815 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:08.865601 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:08.865622 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:08.867404 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:08.869583 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:08.869590 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:08.869609 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:08.870043 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:08.870373 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:08.870376 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:08.870378 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:08.870603 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:08.871067 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:08.871074 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:08.871469 (+ 395us) client_negotiation.cc:769] Sending connection context
0619 02:32:08.871702 (+ 233us) client_negotiation.cc:240] Negotiation successful
0619 02:32:08.872069 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5890,"thread_start_us":168,"threads_started":1}
I20260619 02:32:08.874639 30532 heartbeater.cc:347] Connected to a master server at 127.17.96.254:34729
I20260619 02:32:08.874918 30532 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:08.875569 30532 heartbeater.cc:511] Master 127.17.96.254:34729 requested a full tablet report, sending...
I20260619 02:32:08.876916 30048 ts_manager.cc:194] Registered new tserver with Master: 79e08f569c3d42f0ab6abbcdf1636360 (127.17.96.195:41983)
I20260619 02:32:08.877702 30048 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:45203
I20260619 02:32:08.879298 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:32:08.880160 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 30129
I20260619 02:32:08.885279 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 30268
I20260619 02:32:08.891110 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 30403
I20260619 02:32:08.898731 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 30016
I20260619 02:32:08.907850 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:34729
--webserver_interface=127.17.96.254
--webserver_port=40535
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:36967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:34729
--ranger_config_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.96.212:42257/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.96.212:42257/kms with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:32:09.025566 30535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:09.026021 30535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:09.026103 30535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:09.030566 30535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:32:09.030681 30535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:09.030718 30535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:09.030758 30535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:32:09.030793 30535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:32:09.037299 30535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:36967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.96.212:42257/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:34729
--ranger_config_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:34729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=40535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:09.038656 30535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:09.039779 30535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:09.048693 30540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:09.048753 30543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:09.048696 30541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:09.049058 30535 server_base.cc:1034] running on GCE node
I20260619 02:32:09.049646 30535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:09.050899 30535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:09.052109 30535 hybrid_clock.cc:648] HybridClock initialized: now 1781836329052093 us; error 31 us; skew 500 ppm
Jun 19 02:32:09 dist-test-slave-4qw6 krb5kdc[29572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836329, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:09.055085 30535 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:32:09.056089 30535 webserver.cc:460] Webserver started at http://127.17.96.254:40535/ using document root <none> and password file <none>
I20260619 02:32:09.056641 30535 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:09.056710 30535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 02:32:09 dist-test-slave-4qw6 krb5kdc[29572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836329, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for HTTP/127.17.96.212@KRBTEST.COM
I20260619 02:32:09.105691 30535 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 02:32:09.108886 30551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:09.110472 30535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:32:09.110641 30535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "1cc8a20fa02f4b198e68971c2b23c2da"
format_stamp: "Formatted at 2026-06-19 02:32:06 on dist-test-slave-4qw6"
server_key: "411c510efc1e1322531941f3235fb06a"
server_key_iv: "13f54fcecd7826d75bb99b90dcb8e66d"
server_key_version: "kuduclusterkey@0"
I20260619 02:32:09.111274 30535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:09.146174 30535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:09.149631 30535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:09.149916 30535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:09.158543 30535 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:34729
I20260619 02:32:09.158545 30603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:34729 every 8 connection(s)
I20260619 02:32:09.159840 30535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:32:09.164731 30604 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:32:09.166724 30604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da: Bootstrap starting.
I20260619 02:32:09.166874 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 30535
I20260619 02:32:09.167416 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.96.212:42257/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:36749
--local_ip_for_outbound_sockets=127.17.96.193
--tserver_master_addrs=127.17.96.254:34729
--webserver_port=37555
--webserver_interface=127.17.96.193
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:36967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.96.212:42257/kms with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:32:09.190752 30604 log.cc:824] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da: Log is configured to *not* fsync() on all Append() calls
I20260619 02:32:09.200313 30604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:32:09.200878 30604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da: Bootstrap complete.
I20260619 02:32:09.206665 30604 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc8a20fa02f4b198e68971c2b23c2da" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34729 } }
I20260619 02:32:09.207239 30604 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc8a20fa02f4b198e68971c2b23c2da, State: Initialized, Role: FOLLOWER
I20260619 02:32:09.207886 30604 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc8a20fa02f4b198e68971c2b23c2da" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34729 } }
I20260619 02:32:09.208046 30604 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:32:09.208145 30604 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:32:09.208266 30604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:32:09.211170 30604 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc8a20fa02f4b198e68971c2b23c2da" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34729 } }
I20260619 02:32:09.211576 30604 leader_election.cc:302] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cc8a20fa02f4b198e68971c2b23c2da; no voters:
I20260619 02:32:09.211975 30604 leader_election.cc:288] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 02:32:09.212110 30608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [term 2 FOLLOWER]: Leader election won for term 2
I20260619 02:32:09.212455 30608 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [term 2 LEADER]: Becoming Leader. State: Replica: 1cc8a20fa02f4b198e68971c2b23c2da, State: Running, Role: LEADER
I20260619 02:32:09.212978 30608 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc8a20fa02f4b198e68971c2b23c2da" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34729 } }
I20260619 02:32:09.213521 30604 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:32:09.214128 30608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1cc8a20fa02f4b198e68971c2b23c2da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc8a20fa02f4b198e68971c2b23c2da" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34729 } } }
I20260619 02:32:09.214340 30608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [sys.catalog]: This master's current role is: LEADER
I20260619 02:32:09.214583 30608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cc8a20fa02f4b198e68971c2b23c2da. Latest consensus state: current_term: 2 leader_uuid: "1cc8a20fa02f4b198e68971c2b23c2da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc8a20fa02f4b198e68971c2b23c2da" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34729 } } }
I20260619 02:32:09.214709 30608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da [sys.catalog]: This master's current role is: LEADER
I20260619 02:32:09.215035 30613 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:32:09.218829 30613 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:32:09.219422 30604 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 02:32:09.219743 30613 catalog_manager.cc:1259] Loaded cluster ID: 7a085fee84814aed80e9b274a0cae196
I20260619 02:32:09.219851 30613 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:32:09.222570 30613 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:32:09.223541 30613 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da: Loaded TSK: 0
I20260619 02:32:09.224931 30613 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:32:09.334265 30606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:09.334769 30606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:09.334900 30606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:09.341536 30606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:09.341737 30606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:09.341909 30606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:32:09.350508 30606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:36967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.96.212:42257/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:36749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=37555
--tserver_master_addrs=127.17.96.254:34729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:09.352214 30606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:09.353670 30606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:09.367682 30642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:09.368968 30606 server_base.cc:1034] running on GCE node
W20260619 02:32:09.369315 30644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:09.369510 30641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:09.370162 30606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:09.371057 30606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:09.374358 30606 hybrid_clock.cc:648] HybridClock initialized: now 1781836329374273 us; error 97 us; skew 500 ppm
Jun 19 02:32:09 dist-test-slave-4qw6 krb5kdc[29572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836329, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:09.382781 30606 init.cc:377] Logged in from keytab as kudu/127.17.96.193@KRBTEST.COM (short username kudu)
I20260619 02:32:09.385576 30606 webserver.cc:460] Webserver started at http://127.17.96.193:37555/ using document root <none> and password file <none>
I20260619 02:32:09.388239 30606 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:09.388479 30606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 02:32:09 dist-test-slave-4qw6 krb5kdc[29572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836329, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for HTTP/127.17.96.212@KRBTEST.COM
I20260619 02:32:09.413792 30606 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.004s sys 0.000s
I20260619 02:32:09.416960 30652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:09.418740 30606 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:32:09.418890 30606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "ce896a6be62b4a9d9df161920546a682"
format_stamp: "Formatted at 2026-06-19 02:32:07 on dist-test-slave-4qw6"
server_key: "fc2b3078c3cc78949825c758efac3e52"
server_key_iv: "1539d826b1ad46f7eb0fc817807df9be"
server_key_version: "kuduclusterkey@0"
I20260619 02:32:09.419679 30606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:09.450703 30606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:09.454864 30606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:09.455083 30606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:09.455786 30606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:32:09.456877 30606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:32:09.456940 30606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:09.456992 30606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:32:09.457070 30606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:09.468957 30606 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:36749
I20260619 02:32:09.469024 30765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:36749 every 8 connection(s)
I20260619 02:32:09.470628 30606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:32:09.474012 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 30606
I20260619 02:32:09.474541 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.96.212:42257/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:33721
--local_ip_for_outbound_sockets=127.17.96.194
--tserver_master_addrs=127.17.96.254:34729
--webserver_port=40923
--webserver_interface=127.17.96.194
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:36967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.96.212:42257/kms with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:32:09 dist-test-slave-4qw6 krb5kdc[29572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836329, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:09.501434 30769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:09.480706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:34729 (local address 127.17.96.193:38389)
0619 02:32:09.481395 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:09.481466 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:09.482609 (+ 1143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:09.485213 (+ 2604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:09.485227 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:09.485663 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:09.486462 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:09.486487 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:09.488777 (+ 2290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:09.488784 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:09.490420 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:09.490436 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:09.491783 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:09.491789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:09.491977 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:09.492731 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:09.492752 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:09.494852 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:09.497572 (+ 2720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:09.497581 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:09.497599 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:09.498008 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:09.498436 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:09.498441 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:09.498445 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:09.498625 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:09.499443 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:09.499450 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:09.499758 (+ 308us) client_negotiation.cc:769] Sending connection context
0619 02:32:09.499950 (+ 192us) client_negotiation.cc:240] Negotiation successful
0619 02:32:09.500330 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":190,"threads_started":1}
I20260619 02:32:09.503536 30770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:09.482401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:38389 (local address 127.17.96.254:34729)
0619 02:32:09.483169 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:09.483194 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:09.483245 (+ 51us) server_negotiation.cc:407] Connection header received
0619 02:32:09.483607 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:09.483630 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:09.484130 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:09.484579 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:09.486739 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:09.488561 (+ 1822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:09.490601 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:09.491353 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:09.491522 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:09.495123 (+ 3601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:09.495131 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:09.495150 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:09.495186 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:09.497378 (+ 2192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:09.498165 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:09.498174 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:09.498179 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:09.498332 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:09.498732 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:09.498736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:09.498738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:09.499255 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:09.499389 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:09.502146 (+ 2757us) server_negotiation.cc:299] Negotiation successful
0619 02:32:09.502497 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":155,"threads_started":1}
I20260619 02:32:09.507295 30766 heartbeater.cc:347] Connected to a master server at 127.17.96.254:34729
I20260619 02:32:09.507570 30766 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:09.508170 30766 heartbeater.cc:511] Master 127.17.96.254:34729 requested a full tablet report, sending...
W20260619 02:32:09.510685 30766 heartbeater.cc:644] Failed to heartbeat to 127.17.96.254:34729 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 02:32:09.510815 30766 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:09.511005 30766 heartbeater.cc:511] Master 127.17.96.254:34729 requested a full tablet report, sending...
I20260619 02:32:09.512146 30766 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:09.512331 30766 heartbeater.cc:511] Master 127.17.96.254:34729 requested a full tablet report, sending...
W20260619 02:32:09.513216 30766 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:32:09.651688 30768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:09.652249 30768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:09.652396 30768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:09.659353 30768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:09.659541 30768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:09.659691 30768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:32:09.668569 30768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:36967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.96.212:42257/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:33721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=40923
--tserver_master_addrs=127.17.96.254:34729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:09.670141 30768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:09.671573 30768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:09.684128 30778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:09.684288 30775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:09.684128 30776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:09.684631 30768 server_base.cc:1034] running on GCE node
I20260619 02:32:09.685302 30768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:09.686134 30768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:09.687374 30768 hybrid_clock.cc:648] HybridClock initialized: now 1781836329687355 us; error 31 us; skew 500 ppm
Jun 19 02:32:09 dist-test-slave-4qw6 krb5kdc[29572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836329, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:09.691336 30768 init.cc:377] Logged in from keytab as kudu/127.17.96.194@KRBTEST.COM (short username kudu)
I20260619 02:32:09.692662 30768 webserver.cc:460] Webserver started at http://127.17.96.194:40923/ using document root <none> and password file <none>
I20260619 02:32:09.693499 30768 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:09.693617 30768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 02:32:09 dist-test-slave-4qw6 krb5kdc[29572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836329, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for HTTP/127.17.96.212@KRBTEST.COM
I20260619 02:32:09.716080 30768 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 02:32:09.719182 30786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:09.720886 30768 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 02:32:09.721056 30768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "861fd6bbc07b4da6bf07f2848d5d398d"
format_stamp: "Formatted at 2026-06-19 02:32:08 on dist-test-slave-4qw6"
server_key: "1161388b66fee8a1d72520aea8932c42"
server_key_iv: "a857eb5cd502623fb6dcd5cf4a736942"
server_key_version: "kuduclusterkey@0"
I20260619 02:32:09.721899 30768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:09.764367 30768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:09.767746 30768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:09.767993 30768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:09.768728 30768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:32:09.769701 30768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:32:09.769797 30768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:09.769898 30768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:32:09.769953 30768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:09.783726 30768 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:33721
I20260619 02:32:09.783871 30899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:33721 every 8 connection(s)
I20260619 02:32:09.785037 30768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:32:09.786736 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 30768
I20260619 02:32:09.787209 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.96.212:42257/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:41983
--local_ip_for_outbound_sockets=127.17.96.195
--tserver_master_addrs=127.17.96.254:34729
--webserver_port=42817
--webserver_interface=127.17.96.195
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:36967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.96.212:42257/kms with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:32:09 dist-test-slave-4qw6 krb5kdc[29572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836329, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:09.802111 30770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:09.788091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:35123 (local address 127.17.96.254:34729)
0619 02:32:09.788260 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:09.788265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:09.789914 (+ 1649us) server_negotiation.cc:407] Connection header received
0619 02:32:09.791240 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:09.791244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:09.791317 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:09.791452 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:09.792686 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:09.793331 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:09.794638 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:09.794990 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:09.795124 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:09.798068 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:09.798072 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:09.798074 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:09.798101 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:09.799626 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:09.800251 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:09.800255 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:09.800257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:09.800356 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:09.800803 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:09.800807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:09.800808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:09.801066 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:09.801156 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:09.801744 (+ 588us) server_negotiation.cc:299] Negotiation successful
0619 02:32:09.801925 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:32:09.803048 30903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:09.788640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:34729 (local address 127.17.96.194:35123)
0619 02:32:09.789648 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:09.789721 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:09.790928 (+ 1207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:09.791672 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:09.791679 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:09.791971 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:09.792511 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:09.792525 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:09.793469 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:09.793473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:09.794490 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:09.794500 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:09.795320 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:09.795325 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:09.795552 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:09.796346 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:09.796368 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:09.797865 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:09.799759 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:09.799766 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:09.799785 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:09.800121 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:09.800473 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:09.800476 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:09.800478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:09.800684 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:09.801215 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:09.801220 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:09.801485 (+ 265us) client_negotiation.cc:769] Sending connection context
0619 02:32:09.801692 (+ 207us) client_negotiation.cc:240] Negotiation successful
0619 02:32:09.802071 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":162,"threads_started":1}
I20260619 02:32:09.804358 30900 heartbeater.cc:347] Connected to a master server at 127.17.96.254:34729
I20260619 02:32:09.804658 30900 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:09.805190 30900 heartbeater.cc:511] Master 127.17.96.254:34729 requested a full tablet report, sending...
W20260619 02:32:09.806865 30900 heartbeater.cc:644] Failed to heartbeat to 127.17.96.254:34729 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 02:32:09.807014 30900 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:09.807216 30900 heartbeater.cc:511] Master 127.17.96.254:34729 requested a full tablet report, sending...
I20260619 02:32:09.808096 30900 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:09.808313 30900 heartbeater.cc:511] Master 127.17.96.254:34729 requested a full tablet report, sending...
W20260619 02:32:09.809118 30900 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:32:09.941216 30902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:09.941635 30902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:09.941741 30902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:09.947300 30902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:09.947403 30902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:09.947499 30902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:32:09.954101 30902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:36967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.96.212:42257/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:41983
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=42817
--tserver_master_addrs=127.17.96.254:34729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:09.955277 30902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:09.956403 30902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:09.967182 30909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:09.967185 30910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:09.967517 30902 server_base.cc:1034] running on GCE node
W20260619 02:32:09.967262 30912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:09.968288 30902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:09.969135 30902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:09.972437 30902 hybrid_clock.cc:648] HybridClock initialized: now 1781836329972215 us; error 235 us; skew 500 ppm
Jun 19 02:32:09 dist-test-slave-4qw6 krb5kdc[29572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836329, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:09.976187 30902 init.cc:377] Logged in from keytab as kudu/127.17.96.195@KRBTEST.COM (short username kudu)
I20260619 02:32:09.977500 30902 webserver.cc:460] Webserver started at http://127.17.96.195:42817/ using document root <none> and password file <none>
I20260619 02:32:09.978335 30902 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:09.978441 30902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 02:32:09 dist-test-slave-4qw6 krb5kdc[29572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836329, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for HTTP/127.17.96.212@KRBTEST.COM
I20260619 02:32:10.004889 30902 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 02:32:10.008361 30920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:10.010354 30902 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 02:32:10.010537 30902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "79e08f569c3d42f0ab6abbcdf1636360"
format_stamp: "Formatted at 2026-06-19 02:32:08 on dist-test-slave-4qw6"
server_key: "18f8abba608a2bc0233b9d92e665473e"
server_key_iv: "7f5a54ff49b3fc9dc1b0657a4588e41a"
server_key_version: "kuduclusterkey@0"
I20260619 02:32:10.011410 30902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:10.038666 30902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:10.042577 30902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:10.042821 30902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:10.043579 30902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:32:10.044564 30902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:32:10.044615 30902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:10.044658 30902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:32:10.044718 30902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:10.057832 30902 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:41983
I20260619 02:32:10.059389 30902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:32:10.065955 31033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:41983 every 8 connection(s)
I20260619 02:32:10.068859 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 30902
Jun 19 02:32:10 dist-test-slave-4qw6 krb5kdc[29572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836329, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:10.080945 30770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:10.062236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:50353 (local address 127.17.96.254:34729)
0619 02:32:10.062388 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:10.062393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:10.063677 (+ 1284us) server_negotiation.cc:407] Connection header received
0619 02:32:10.065049 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:10.065053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:10.065121 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:10.065235 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:10.066963 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:10.067801 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:10.070148 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:10.070493 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:10.070579 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:10.074446 (+ 3867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:10.074451 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:10.074454 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:10.074486 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:10.077708 (+ 3222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:10.078525 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:10.078529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:10.078531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:10.078642 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:10.079189 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:10.079193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:10.079195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:10.079442 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:10.079531 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:10.080575 (+ 1044us) server_negotiation.cc:299] Negotiation successful
0619 02:32:10.080767 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 02:32:10.083127 31036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:10.062817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:34729 (local address 127.17.96.195:50353)
0619 02:32:10.063456 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:10.063521 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:10.064734 (+ 1213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:10.065465 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:10.065475 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:10.065923 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:10.066757 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:10.066782 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:10.067977 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:10.067981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:10.069896 (+ 1915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:10.069911 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:10.070841 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:10.070845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:10.071095 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:10.071762 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:10.071781 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:10.074204 (+ 2423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:10.077919 (+ 3715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:10.077929 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:10.077952 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:10.078388 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:10.078839 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:10.078842 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:10.078844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:10.079060 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:10.079624 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:10.079632 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:10.080004 (+ 372us) client_negotiation.cc:769] Sending connection context
0619 02:32:10.081468 (+ 1464us) client_negotiation.cc:240] Negotiation successful
0619 02:32:10.081931 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":172,"threads_started":1}
I20260619 02:32:10.084900 31034 heartbeater.cc:347] Connected to a master server at 127.17.96.254:34729
I20260619 02:32:10.085178 31034 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:10.085683 31034 heartbeater.cc:511] Master 127.17.96.254:34729 requested a full tablet report, sending...
W20260619 02:32:10.087602 31034 heartbeater.cc:644] Failed to heartbeat to 127.17.96.254:34729 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 02:32:10.087769 31034 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:10.087975 31034 heartbeater.cc:511] Master 127.17.96.254:34729 requested a full tablet report, sending...
I20260619 02:32:10.088961 31034 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:10.089159 31034 heartbeater.cc:511] Master 127.17.96.254:34729 requested a full tablet report, sending...
W20260619 02:32:10.090068 31034 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 02:32:10.099186 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:10.069654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59902 (local address 127.17.96.254:34729)
0619 02:32:10.070766 (+ 1112us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:10.070777 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:10.070790 (+ 13us) server_negotiation.cc:407] Connection header received
0619 02:32:10.070823 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:10.070825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:10.070868 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:10.070959 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:10.071950 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:10.074510 (+ 2560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:10.090897 (+ 16387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:10.091177 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:10.091312 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:10.095578 (+ 4266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:10.095582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:10.095585 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:10.095626 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:10.097255 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:10.097781 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:10.097785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:10.097787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:10.097894 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:10.098205 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:10.098208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:10.098210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:10.098425 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:10.098670 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:10.098826 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 02:32:10.098971 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1035,"thread_start_us":143,"threads_started":1}
Jun 19 02:32:10 dist-test-slave-4qw6 krb5kdc[29572](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781836330, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:10.513475 30766 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:10.513854 30766 heartbeater.cc:511] Master 127.17.96.254:34729 requested a full tablet report, sending...
I20260619 02:32:10.515273 30568 ts_manager.cc:194] Registered new tserver with Master: ce896a6be62b4a9d9df161920546a682 (127.17.96.193:36749)
I20260619 02:32:10.516777 30568 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:38389
I20260619 02:32:10.809434 30900 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:10.809831 30900 heartbeater.cc:511] Master 127.17.96.254:34729 requested a full tablet report, sending...
I20260619 02:32:10.810854 30568 ts_manager.cc:194] Registered new tserver with Master: 861fd6bbc07b4da6bf07f2848d5d398d (127.17.96.194:33721)
I20260619 02:32:10.812321 30568 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:35123
Jun 19 02:32:10 dist-test-slave-4qw6 krb5kdc[29572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836330, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for HTTP/127.17.96.212@KRBTEST.COM
I20260619 02:32:11.090333 31034 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:11.090750 31034 heartbeater.cc:511] Master 127.17.96.254:34729 requested a full tablet report, sending...
I20260619 02:32:11.091874 30568 ts_manager.cc:194] Registered new tserver with Master: 79e08f569c3d42f0ab6abbcdf1636360 (127.17.96.195:41983)
I20260619 02:32:11.092664 30568 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:50353
I20260619 02:32:11.100425 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:32:11.111299 31055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:11.102458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59908 (local address 127.17.96.254:34729)
0619 02:32:11.103238 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:11.103244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:11.103260 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:32:11.103306 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:11.103309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:11.103363 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:11.103465 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:11.104208 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.104787 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.105641 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.105978 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.106093 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:11.107609 (+ 1516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:11.107613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:11.107616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:11.107655 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:11.109295 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:11.109790 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:11.109795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:11.109797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:11.109895 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:11.110203 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:11.110209 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:11.110214 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:11.110426 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:11.110568 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:11.110838 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 02:32:11.110981 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":695,"thread_start_us":95,"threads_started":1}
I20260619 02:32:11.120023 30568 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59908:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:32:11.125824 30568 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:32:11.167436 31062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:11.148180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:33721 (local address 127.0.0.1:51324)
0619 02:32:11.154625 (+ 6445us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:11.154640 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:11.154735 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:11.156741 (+ 2006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:11.156745 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:11.156769 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:32:11.156887 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:11.156895 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.161340 (+ 4445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.161343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:11.162556 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:11.162571 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.166665 (+ 4094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.166672 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:11.166854 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:11.166919 (+ 65us) client_negotiation.cc:769] Sending connection context
0619 02:32:11.167067 (+ 148us) client_negotiation.cc:240] Negotiation successful
0619 02:32:11.167252 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6370,"thread_start_us":106,"threads_started":1}
I20260619 02:32:11.169418 31067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:11.148531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51324 (local address 127.17.96.194:33721)
0619 02:32:11.151544 (+ 3013us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:11.151550 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:11.154844 (+ 3294us) server_negotiation.cc:407] Connection header received
0619 02:32:11.154946 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:11.154951 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:11.155169 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:11.155341 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:32:11.158987 (+ 3646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.160249 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.165094 (+ 4845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.166220 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.166361 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:11.166561 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:11.169094 (+ 2533us) server_negotiation.cc:299] Negotiation successful
0619 02:32:11.169259 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2896,"thread_start_us":68,"threads_started":1}
I20260619 02:32:11.170137 31063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:11.144919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45182 (local address 127.17.96.195:41983)
0619 02:32:11.150126 (+ 5207us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:11.150135 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:11.152365 (+ 2230us) server_negotiation.cc:407] Connection header received
0619 02:32:11.152470 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:11.152475 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:11.152673 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:11.155508 (+ 2835us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:32:11.155693 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.156915 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.162752 (+ 5837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.167928 (+ 5176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.168083 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:11.168334 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:11.169830 (+ 1496us) server_negotiation.cc:299] Negotiation successful
0619 02:32:11.169974 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5064,"thread_start_us":157,"threads_started":1}
I20260619 02:32:11.168939 31064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:11.144092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:41983 (local address 127.0.0.1:45182)
0619 02:32:11.150457 (+ 6365us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:11.150534 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:11.150694 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:11.154302 (+ 3608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:11.154305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:11.154322 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:32:11.154405 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:11.154410 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.157087 (+ 2677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.157093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:11.161176 (+ 4083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:11.161203 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.168431 (+ 7228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.168435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:11.168545 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:11.168575 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 02:32:11.168657 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 02:32:11.168766 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6142,"thread_start_us":134,"threads_started":1}
I20260619 02:32:11.172225 31066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:11.146816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:36749 (local address 127.0.0.1:33770)
0619 02:32:11.151155 (+ 4339us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:11.151170 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:11.151280 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:11.153955 (+ 2675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:11.153964 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:11.153993 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:32:11.154162 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:11.154171 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.158590 (+ 4419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.158594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:11.159844 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:11.159864 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.171458 (+ 11594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.171464 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:11.171579 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:11.171608 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 02:32:11.171699 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 02:32:11.172065 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4275,"thread_start_us":115,"threads_started":1}
I20260619 02:32:11.173075 31065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:11.147299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33770 (local address 127.17.96.193:36749)
0619 02:32:11.151013 (+ 3714us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:11.151020 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:11.151607 (+ 587us) server_negotiation.cc:407] Connection header received
0619 02:32:11.151710 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:11.151716 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:11.151997 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:11.152169 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:32:11.157103 (+ 4934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.158405 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.170020 (+ 11615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.171045 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.171188 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:11.171364 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:11.172756 (+ 1392us) server_negotiation.cc:299] Negotiation successful
0619 02:32:11.172895 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3593,"thread_start_us":134,"threads_started":1}
I20260619 02:32:11.176198 30816 tablet_service.cc:1467] Processing CreateTablet for tablet 683a01f5ff3f4d2b83d9498c03fb5d3e (DEFAULT_TABLE table=test-table [id=eea52188f8664c2285eceddb38c0d038]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:32:11.177593 30816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 683a01f5ff3f4d2b83d9498c03fb5d3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:32:11.181511 30968 tablet_service.cc:1467] Processing CreateTablet for tablet 683a01f5ff3f4d2b83d9498c03fb5d3e (DEFAULT_TABLE table=test-table [id=eea52188f8664c2285eceddb38c0d038]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:32:11.182835 30968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 683a01f5ff3f4d2b83d9498c03fb5d3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:32:11.184269 30700 tablet_service.cc:1467] Processing CreateTablet for tablet 683a01f5ff3f4d2b83d9498c03fb5d3e (DEFAULT_TABLE table=test-table [id=eea52188f8664c2285eceddb38c0d038]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:32:11.185432 30700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 683a01f5ff3f4d2b83d9498c03fb5d3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:32:11.188628 31068 tablet_bootstrap.cc:492] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 861fd6bbc07b4da6bf07f2848d5d398d: Bootstrap starting.
I20260619 02:32:11.191291 31068 tablet_bootstrap.cc:654] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 861fd6bbc07b4da6bf07f2848d5d398d: Neither blocks nor log segments found. Creating new log.
I20260619 02:32:11.192369 31068 log.cc:824] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 861fd6bbc07b4da6bf07f2848d5d398d: Log is configured to *not* fsync() on all Append() calls
I20260619 02:32:11.193142 31069 tablet_bootstrap.cc:492] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 79e08f569c3d42f0ab6abbcdf1636360: Bootstrap starting.
I20260619 02:32:11.195748 31068 tablet_bootstrap.cc:492] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 861fd6bbc07b4da6bf07f2848d5d398d: No bootstrap required, opened a new log
I20260619 02:32:11.195986 31068 ts_tablet_manager.cc:1397] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 861fd6bbc07b4da6bf07f2848d5d398d: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260619 02:32:11.196014 31069 tablet_bootstrap.cc:654] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 79e08f569c3d42f0ab6abbcdf1636360: Neither blocks nor log segments found. Creating new log.
I20260619 02:32:11.196890 31069 log.cc:824] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 79e08f569c3d42f0ab6abbcdf1636360: Log is configured to *not* fsync() on all Append() calls
I20260619 02:32:11.199828 31069 tablet_bootstrap.cc:492] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 79e08f569c3d42f0ab6abbcdf1636360: No bootstrap required, opened a new log
I20260619 02:32:11.200100 31069 ts_tablet_manager.cc:1397] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 79e08f569c3d42f0ab6abbcdf1636360: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260619 02:32:11.200572 31068 raft_consensus.cc:348] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 861fd6bbc07b4da6bf07f2848d5d398d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e08f569c3d42f0ab6abbcdf1636360" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 41983 } } peers { permanent_uuid: "ce896a6be62b4a9d9df161920546a682" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 36749 } } peers { permanent_uuid: "861fd6bbc07b4da6bf07f2848d5d398d" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33721 } }
I20260619 02:32:11.200889 31068 raft_consensus.cc:374] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 861fd6bbc07b4da6bf07f2848d5d398d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:32:11.200947 31068 raft_consensus.cc:729] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 861fd6bbc07b4da6bf07f2848d5d398d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 861fd6bbc07b4da6bf07f2848d5d398d, State: Initialized, Role: FOLLOWER
I20260619 02:32:11.201509 31068 consensus_queue.cc:260] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 861fd6bbc07b4da6bf07f2848d5d398d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e08f569c3d42f0ab6abbcdf1636360" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 41983 } } peers { permanent_uuid: "ce896a6be62b4a9d9df161920546a682" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 36749 } } peers { permanent_uuid: "861fd6bbc07b4da6bf07f2848d5d398d" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33721 } }
I20260619 02:32:11.202754 31068 ts_tablet_manager.cc:1428] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 861fd6bbc07b4da6bf07f2848d5d398d: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260619 02:32:11.203186 30900 heartbeater.cc:503] Master 127.17.96.254:34729 was elected leader, sending a full tablet report...
I20260619 02:32:11.205884 31069 raft_consensus.cc:348] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 79e08f569c3d42f0ab6abbcdf1636360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e08f569c3d42f0ab6abbcdf1636360" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 41983 } } peers { permanent_uuid: "ce896a6be62b4a9d9df161920546a682" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 36749 } } peers { permanent_uuid: "861fd6bbc07b4da6bf07f2848d5d398d" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33721 } }
I20260619 02:32:11.206302 31069 raft_consensus.cc:374] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 79e08f569c3d42f0ab6abbcdf1636360 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:32:11.206421 31069 raft_consensus.cc:729] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 79e08f569c3d42f0ab6abbcdf1636360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e08f569c3d42f0ab6abbcdf1636360, State: Initialized, Role: FOLLOWER
I20260619 02:32:11.207005 31069 consensus_queue.cc:260] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 79e08f569c3d42f0ab6abbcdf1636360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e08f569c3d42f0ab6abbcdf1636360" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 41983 } } peers { permanent_uuid: "ce896a6be62b4a9d9df161920546a682" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 36749 } } peers { permanent_uuid: "861fd6bbc07b4da6bf07f2848d5d398d" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33721 } }
I20260619 02:32:11.207922 31073 tablet_bootstrap.cc:492] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682: Bootstrap starting.
I20260619 02:32:11.208328 31069 ts_tablet_manager.cc:1428] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 79e08f569c3d42f0ab6abbcdf1636360: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260619 02:32:11.208493 31034 heartbeater.cc:503] Master 127.17.96.254:34729 was elected leader, sending a full tablet report...
I20260619 02:32:11.212678 31073 tablet_bootstrap.cc:654] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682: Neither blocks nor log segments found. Creating new log.
I20260619 02:32:11.213553 31073 log.cc:824] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682: Log is configured to *not* fsync() on all Append() calls
I20260619 02:32:11.216609 31073 tablet_bootstrap.cc:492] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682: No bootstrap required, opened a new log
I20260619 02:32:11.216852 31073 ts_tablet_manager.cc:1397] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260619 02:32:11.221010 31073 raft_consensus.cc:348] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e08f569c3d42f0ab6abbcdf1636360" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 41983 } } peers { permanent_uuid: "ce896a6be62b4a9d9df161920546a682" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 36749 } } peers { permanent_uuid: "861fd6bbc07b4da6bf07f2848d5d398d" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33721 } }
I20260619 02:32:11.221472 31073 raft_consensus.cc:374] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:32:11.221596 31073 raft_consensus.cc:729] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce896a6be62b4a9d9df161920546a682, State: Initialized, Role: FOLLOWER
I20260619 02:32:11.222522 31073 consensus_queue.cc:260] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e08f569c3d42f0ab6abbcdf1636360" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 41983 } } peers { permanent_uuid: "ce896a6be62b4a9d9df161920546a682" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 36749 } } peers { permanent_uuid: "861fd6bbc07b4da6bf07f2848d5d398d" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33721 } }
I20260619 02:32:11.223920 31073 ts_tablet_manager.cc:1428] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260619 02:32:11.224409 30766 heartbeater.cc:503] Master 127.17.96.254:34729 was elected leader, sending a full tablet report...
W20260619 02:32:11.229686 30767 tablet.cc:2369] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:32:11.272958 31076 raft_consensus.cc:482] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:32:11.273387 31076 raft_consensus.cc:504] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e08f569c3d42f0ab6abbcdf1636360" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 41983 } } peers { permanent_uuid: "ce896a6be62b4a9d9df161920546a682" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 36749 } } peers { permanent_uuid: "861fd6bbc07b4da6bf07f2848d5d398d" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33721 } }
I20260619 02:32:11.275380 31076 leader_election.cc:288] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e08f569c3d42f0ab6abbcdf1636360 (127.17.96.195:41983), 861fd6bbc07b4da6bf07f2848d5d398d (127.17.96.194:33721)
I20260619 02:32:11.280966 31063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:11.275889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:39657 (local address 127.17.96.195:41983)
0619 02:32:11.276062 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:11.276068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:11.276219 (+ 151us) server_negotiation.cc:407] Connection header received
0619 02:32:11.276441 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:11.276445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:11.276493 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:11.276548 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:32:11.277024 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.277846 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.279528 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.280175 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.280287 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:11.280365 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:11.280729 (+ 364us) server_negotiation.cc:299] Negotiation successful
0619 02:32:11.280818 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 02:32:11.281675 31078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:11.275348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:41983 (local address 127.17.96.193:39657)
0619 02:32:11.276115 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:11.276132 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:11.276311 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:11.276699 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:11.276702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:11.276780 (+ 78us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:32:11.276878 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:11.276884 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.278011 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.278013 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:11.279352 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:11.279367 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.280470 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.280473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:11.280571 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:11.280599 (+ 28us) client_negotiation.cc:769] Sending connection context
0619 02:32:11.281422 (+ 823us) client_negotiation.cc:240] Negotiation successful
0619 02:32:11.281510 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"mutex_wait_us":8,"spinlock_wait_cycles":22272,"thread_start_us":161,"threads_started":1}
I20260619 02:32:11.282573 30975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "683a01f5ff3f4d2b83d9498c03fb5d3e" candidate_uuid: "ce896a6be62b4a9d9df161920546a682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e08f569c3d42f0ab6abbcdf1636360" is_pre_election: true
I20260619 02:32:11.282936 30975 raft_consensus.cc:2454] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 79e08f569c3d42f0ab6abbcdf1636360 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce896a6be62b4a9d9df161920546a682 in term 0.
I20260619 02:32:11.283491 30653 leader_election.cc:302] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e08f569c3d42f0ab6abbcdf1636360, ce896a6be62b4a9d9df161920546a682; no voters:
I20260619 02:32:11.283759 31067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:11.275987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:48057 (local address 127.17.96.194:33721)
0619 02:32:11.276182 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:11.276200 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:11.276267 (+ 67us) server_negotiation.cc:407] Connection header received
0619 02:32:11.276441 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:11.276445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:11.276493 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:11.276555 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:32:11.277014 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.277846 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.282428 (+ 4582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.283014 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.283118 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:11.283171 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:11.283552 (+ 381us) server_negotiation.cc:299] Negotiation successful
0619 02:32:11.283644 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260619 02:32:11.283839 31076 raft_consensus.cc:2781] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:32:11.283921 31076 raft_consensus.cc:482] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:32:11.283968 31076 raft_consensus.cc:3037] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:32:11.284214 31077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:11.275348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:33721 (local address 127.17.96.193:48057)
0619 02:32:11.276187 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:11.276202 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:11.276314 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:11.276699 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:11.276702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:11.276720 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:32:11.276877 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:11.276884 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.278006 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.278010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:11.281306 (+ 3296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:11.281317 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.283280 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.283284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:11.283385 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:11.283407 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 02:32:11.283950 (+ 543us) client_negotiation.cc:240] Negotiation successful
0619 02:32:11.284044 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":768,"mutex_wait_us":2299,"thread_start_us":81,"threads_started":1}
I20260619 02:32:11.285410 31076 raft_consensus.cc:504] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e08f569c3d42f0ab6abbcdf1636360" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 41983 } } peers { permanent_uuid: "ce896a6be62b4a9d9df161920546a682" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 36749 } } peers { permanent_uuid: "861fd6bbc07b4da6bf07f2848d5d398d" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33721 } }
I20260619 02:32:11.285887 31076 leader_election.cc:288] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682 [CANDIDATE]: Term 1 election: Requested vote from peers 79e08f569c3d42f0ab6abbcdf1636360 (127.17.96.195:41983), 861fd6bbc07b4da6bf07f2848d5d398d (127.17.96.194:33721)
I20260619 02:32:11.286326 30844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "683a01f5ff3f4d2b83d9498c03fb5d3e" candidate_uuid: "ce896a6be62b4a9d9df161920546a682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "861fd6bbc07b4da6bf07f2848d5d398d" is_pre_election: true
I20260619 02:32:11.286664 30844 raft_consensus.cc:2454] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 861fd6bbc07b4da6bf07f2848d5d398d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce896a6be62b4a9d9df161920546a682 in term 0.
W20260619 02:32:11.287518 30901 tablet.cc:2369] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 861fd6bbc07b4da6bf07f2848d5d398d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:32:11.288652 30844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "683a01f5ff3f4d2b83d9498c03fb5d3e" candidate_uuid: "ce896a6be62b4a9d9df161920546a682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "861fd6bbc07b4da6bf07f2848d5d398d"
I20260619 02:32:11.288812 30844 raft_consensus.cc:3037] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 861fd6bbc07b4da6bf07f2848d5d398d [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:32:11.289417 30975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "683a01f5ff3f4d2b83d9498c03fb5d3e" candidate_uuid: "ce896a6be62b4a9d9df161920546a682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e08f569c3d42f0ab6abbcdf1636360"
I20260619 02:32:11.289551 30975 raft_consensus.cc:3037] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 79e08f569c3d42f0ab6abbcdf1636360 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:32:11.290637 30844 raft_consensus.cc:2454] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 861fd6bbc07b4da6bf07f2848d5d398d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce896a6be62b4a9d9df161920546a682 in term 1.
I20260619 02:32:11.291178 30654 leader_election.cc:302] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 861fd6bbc07b4da6bf07f2848d5d398d, ce896a6be62b4a9d9df161920546a682; no voters:
I20260619 02:32:11.291323 30975 raft_consensus.cc:2454] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 79e08f569c3d42f0ab6abbcdf1636360 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce896a6be62b4a9d9df161920546a682 in term 1.
I20260619 02:32:11.291807 31076 raft_consensus.cc:2781] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:32:11.292078 31076 raft_consensus.cc:686] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682 [term 1 LEADER]: Becoming Leader. State: Replica: ce896a6be62b4a9d9df161920546a682, State: Running, Role: LEADER
I20260619 02:32:11.292484 31076 consensus_queue.cc:237] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e08f569c3d42f0ab6abbcdf1636360" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 41983 } } peers { permanent_uuid: "ce896a6be62b4a9d9df161920546a682" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 36749 } } peers { permanent_uuid: "861fd6bbc07b4da6bf07f2848d5d398d" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33721 } }
I20260619 02:32:11.297359 30563 catalog_manager.cc:5495] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce896a6be62b4a9d9df161920546a682 (127.17.96.193). New cstate: current_term: 1 leader_uuid: "ce896a6be62b4a9d9df161920546a682" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e08f569c3d42f0ab6abbcdf1636360" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 41983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce896a6be62b4a9d9df161920546a682" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 36749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "861fd6bbc07b4da6bf07f2848d5d398d" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33721 } health_report { overall_health: UNKNOWN } } }
Jun 19 02:32:11 dist-test-slave-4qw6 krb5kdc[29572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836330, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for HTTP/127.17.96.212@KRBTEST.COM
W20260619 02:32:11.313110 31035 tablet.cc:2369] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 79e08f569c3d42f0ab6abbcdf1636360: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:32:11.345516 31065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:11.340292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33772 (local address 127.17.96.193:36749)
0619 02:32:11.340462 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:11.340467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:11.340480 (+ 13us) server_negotiation.cc:407] Connection header received
0619 02:32:11.340577 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:11.340580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:11.340647 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:11.340762 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:32:11.341133 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.341938 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.343159 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:11.343504 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:11.343602 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:11.344061 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:32:11.344677 (+ 616us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:32:11.344801 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:11.345276 (+ 475us) server_negotiation.cc:299] Negotiation successful
0619 02:32:11.345348 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 02:32:11.353956 30844 raft_consensus.cc:1261] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 861fd6bbc07b4da6bf07f2848d5d398d [term 1 FOLLOWER]: Refusing update from remote peer ce896a6be62b4a9d9df161920546a682: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:32:11.353956 30975 raft_consensus.cc:1261] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 79e08f569c3d42f0ab6abbcdf1636360 [term 1 FOLLOWER]: Refusing update from remote peer ce896a6be62b4a9d9df161920546a682: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:32:11.355325 31080 consensus_queue.cc:1035] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e08f569c3d42f0ab6abbcdf1636360" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 41983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:32:11.355715 31076 consensus_queue.cc:1035] T 683a01f5ff3f4d2b83d9498c03fb5d3e P ce896a6be62b4a9d9df161920546a682 [LEADER]: Connected to new peer: Peer: permanent_uuid: "861fd6bbc07b4da6bf07f2848d5d398d" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:32:11.372555 31083 mvcc.cc:205] Tried to move back new op lower bound from 7298401613216305152 to 7298401612979240960. Current Snapshot: MvccSnapshot[applied={T|T < 7298401613216305152}]
I20260619 02:32:11.377873 31085 mvcc.cc:205] Tried to move back new op lower bound from 7298401613216305152 to 7298401612979240960. Current Snapshot: MvccSnapshot[applied={T|T < 7298401613216305152}]
I20260619 02:32:11.381287 31086 mvcc.cc:205] Tried to move back new op lower bound from 7298401613216305152 to 7298401612979240960. Current Snapshot: MvccSnapshot[applied={T|T < 7298401613216305152}]
I20260619 02:32:11.395946 30568 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:32:11.396194 30568 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:32:11.401921 30568 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1cc8a20fa02f4b198e68971c2b23c2da: Sending DeleteTablet for 3 replicas of tablet 683a01f5ff3f4d2b83d9498c03fb5d3e
I20260619 02:32:11.403270 30816 tablet_service.cc:1514] Processing DeleteTablet for tablet 683a01f5ff3f4d2b83d9498c03fb5d3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:32:11 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:51324
I20260619 02:32:11.403755 30968 tablet_service.cc:1514] Processing DeleteTablet for tablet 683a01f5ff3f4d2b83d9498c03fb5d3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:32:11 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:45182
I20260619 02:32:11.404812 30700 tablet_service.cc:1514] Processing DeleteTablet for tablet 683a01f5ff3f4d2b83d9498c03fb5d3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:32:11 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:33770
I20260619 02:32:11.406461 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 30606
I20260619 02:32:11.408241 31093 tablet_replica.cc:330] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 861fd6bbc07b4da6bf07f2848d5d398d: stopping tablet replica
I20260619 02:32:11.408660 31093 raft_consensus.cc:2229] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 861fd6bbc07b4da6bf07f2848d5d398d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:32:11.409106 31093 raft_consensus.cc:2258] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 861fd6bbc07b4da6bf07f2848d5d398d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:32:11.412963 31093 ts_tablet_manager.cc:1905] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 861fd6bbc07b4da6bf07f2848d5d398d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:32:11.419106 31094 tablet_replica.cc:330] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 79e08f569c3d42f0ab6abbcdf1636360: stopping tablet replica
I20260619 02:32:11.419682 31094 raft_consensus.cc:2229] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 79e08f569c3d42f0ab6abbcdf1636360 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:32:11.420720 31094 raft_consensus.cc:2258] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 79e08f569c3d42f0ab6abbcdf1636360 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:32:11.420993 31093 ts_tablet_manager.cc:1918] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 861fd6bbc07b4da6bf07f2848d5d398d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:32:11.421257 31093 log.cc:1196] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 861fd6bbc07b4da6bf07f2848d5d398d: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/683a01f5ff3f4d2b83d9498c03fb5d3e
I20260619 02:32:11.421814 31093 ts_tablet_manager.cc:1939] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 861fd6bbc07b4da6bf07f2848d5d398d: Deleting consensus metadata
I20260619 02:32:11.423528 30553 catalog_manager.cc:4841] TS 861fd6bbc07b4da6bf07f2848d5d398d (127.17.96.194:33721): tablet 683a01f5ff3f4d2b83d9498c03fb5d3e (table test-table [id=eea52188f8664c2285eceddb38c0d038]) successfully deleted
I20260619 02:32:11.428853 31094 ts_tablet_manager.cc:1905] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 79e08f569c3d42f0ab6abbcdf1636360: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 02:32:11.435298 30554 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.96.193:36749 (error 108)
I20260619 02:32:11.438489 31094 ts_tablet_manager.cc:1918] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 79e08f569c3d42f0ab6abbcdf1636360: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:32:11.438762 31094 log.cc:1196] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 79e08f569c3d42f0ab6abbcdf1636360: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/683a01f5ff3f4d2b83d9498c03fb5d3e
I20260619 02:32:11.439227 31094 ts_tablet_manager.cc:1939] T 683a01f5ff3f4d2b83d9498c03fb5d3e P 79e08f569c3d42f0ab6abbcdf1636360: Deleting consensus metadata
I20260619 02:32:11.440976 30552 catalog_manager.cc:4841] TS 79e08f569c3d42f0ab6abbcdf1636360 (127.17.96.195:41983): tablet 683a01f5ff3f4d2b83d9498c03fb5d3e (table test-table [id=eea52188f8664c2285eceddb38c0d038]) successfully deleted
I20260619 02:32:11.441988 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 30768
I20260619 02:32:11.446923 31066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:11.446501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:36749 (local address 127.0.0.1:33782)
0619 02:32:11.446728 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:11.446816 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.193:36749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
W20260619 02:32:11.448817 30554 catalog_manager.cc:4584] TS ce896a6be62b4a9d9df161920546a682 (127.17.96.193:36749): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 683a01f5ff3f4d2b83d9498c03fb5d3e: Network error: Client connection negotiation failed: client connection to 127.17.96.193:36749: connect: Connection refused (error 111)
I20260619 02:32:11.452116 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 30902
I20260619 02:32:11.460090 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 30535
I20260619 02:32:11.469058 17795 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 02:32:11.815179 17795 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 02:32:11.815299 17795 mini_ranger.cc:66] Stopping Ranger...
I20260619 02:32:12.160938 17795 mini_ranger.cc:68] Stopped Ranger
2026-06-19 02:32:12.161 UTC [29602] LOG: received smart shutdown request
2026-06-19 02:32:12.165 UTC [29602] LOG: background worker "logical replication launcher" (PID 29611) exited with exit code 1
2026-06-19 02:32:12.165 UTC [29606] LOG: shutting down
2026-06-19 02:32:12.334 UTC [29602] LOG: database system is shut down
2026-06-19T02:32:12Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49845 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:32:12 dist-test-slave-4qw6 krb5kdc[31105](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:32:12 dist-test-slave-4qw6 krb5kdc[31105](info): set up 2 sockets
Jun 19 02:32:12 dist-test-slave-4qw6 krb5kdc[31105](info): commencing operation
krb5kdc: starting...
W20260619 02:32:13.529908 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:32:13 dist-test-slave-4qw6 krb5kdc[31105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:32:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:32:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:32:13.658936 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:35157
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:46481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:35157,127.17.96.253:35101,127.17.96.252:35493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:32:13.775074 31118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:13.775444 31118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:13.775537 31118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:13.779848 31118 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:32:13.779973 31118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:13.780009 31118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:13.780038 31118 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:32:13.780074 31118 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:32:13.785900 31118 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:46481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:35157,127.17.96.253:35101,127.17.96.252:35493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:35157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:13.787160 31118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:13.788281 31118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:13.797997 31126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:13.797999 31124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:13.797999 31123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:13.798089 31118 server_base.cc:1034] running on GCE node
I20260619 02:32:13.798959 31118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:13.800153 31118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:13.801348 31118 hybrid_clock.cc:648] HybridClock initialized: now 1781836333801324 us; error 34 us; skew 500 ppm
Jun 19 02:32:13 dist-test-slave-4qw6 krb5kdc[31105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836333, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:13.804641 31118 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:32:13.805804 31118 webserver.cc:460] Webserver started at http://127.17.96.254:42309/ using document root <none> and password file <none>
I20260619 02:32:13.806458 31118 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:13.806567 31118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:13.806828 31118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:32:13.809092 31118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "597071c01b4f4ecf9c1a3faa21c48495"
format_stamp: "Formatted at 2026-06-19 02:32:13 on dist-test-slave-4qw6"
server_key: "3a38c5ab681fab080423078dcf543916"
server_key_iv: "ed5c7e512379a2aff05905ab25e61ce9"
server_key_version: "encryptionkey@0"
I20260619 02:32:13.809808 31118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "597071c01b4f4ecf9c1a3faa21c48495"
format_stamp: "Formatted at 2026-06-19 02:32:13 on dist-test-slave-4qw6"
server_key: "3a38c5ab681fab080423078dcf543916"
server_key_iv: "ed5c7e512379a2aff05905ab25e61ce9"
server_key_version: "encryptionkey@0"
I20260619 02:32:13.814105 31118 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 02:32:13.816779 31133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:13.818212 31118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:32:13.818382 31118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "597071c01b4f4ecf9c1a3faa21c48495"
format_stamp: "Formatted at 2026-06-19 02:32:13 on dist-test-slave-4qw6"
server_key: "3a38c5ab681fab080423078dcf543916"
server_key_iv: "ed5c7e512379a2aff05905ab25e61ce9"
server_key_version: "encryptionkey@0"
I20260619 02:32:13.818527 31118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:13.834844 31118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:13.838747 31118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:13.838992 31118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:13.846871 31118 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:35157
I20260619 02:32:13.846899 31185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:35157 every 8 connection(s)
I20260619 02:32:13.847904 31118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:32:13.850750 31186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:32:13.854703 31186 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35157 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:32:13.856201 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 31118
I20260619 02:32:13.856314 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:32:13.856600 17795 external_mini_cluster.cc:1428] Setting key 1012ef81423581222e092da7e57e133c
WARNING: no policy specified for kudu/127.17.96.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.253@KRBTEST.COM" created.
Jun 19 02:32:13 dist-test-slave-4qw6 krb5kdc[31105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836333, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:13.873502 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:13.857209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40800 (local address 127.17.96.254:35157)
0619 02:32:13.857817 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:13.857835 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:13.857917 (+ 82us) server_negotiation.cc:407] Connection header received
0619 02:32:13.859010 (+ 1093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:13.859014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:13.859194 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:13.859382 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:13.860244 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:13.861202 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:13.862275 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:13.862656 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:13.862844 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:13.868224 (+ 5380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:13.868231 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:13.868245 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:13.868271 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:13.870137 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:13.870659 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:13.870664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:13.870665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:13.870770 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:13.871097 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:13.871100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:13.871102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:13.871520 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:13.871674 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:13.872041 (+ 367us) server_negotiation.cc:299] Negotiation successful
0619 02:32:13.872372 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":453,"spinlock_wait_cycles":16768,"thread_start_us":177,"threads_started":1}
I20260619 02:32:13.873502 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:13.857209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:35157 (local address 127.0.0.1:40800)
0619 02:32:13.857738 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:13.857815 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:13.858696 (+ 881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:13.859382 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:13.859387 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:13.859845 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:13.860093 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:13.860106 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:13.861340 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:13.861343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:13.862156 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:13.862167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:13.862832 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:13.862836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:13.862949 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:13.863629 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:13.863646 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:13.867994 (+ 4348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:13.870274 (+ 2280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:13.870284 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:13.870295 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:13.870529 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:13.870873 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:13.870876 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:13.870878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:13.870992 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:13.871652 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:13.871658 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:13.871797 (+ 139us) client_negotiation.cc:769] Sending connection context
0619 02:32:13.871970 (+ 173us) client_negotiation.cc:240] Negotiation successful
0619 02:32:13.872422 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":154,"thread_start_us":171,"threads_started":1}
I20260619 02:32:13.876289 31186 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:32:13.877115 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:13.876784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:35101 (local address 127.0.0.1:47760)
0619 02:32:13.876963 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:13.877042 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.253:35101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260619 02:32:13.877439 31134 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.96.253:35101: connect: Connection refused (error 111)
I20260619 02:32:13.878378 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:13.878146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:35101 (local address 127.0.0.1:47762)
0619 02:32:13.878241 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:13.878288 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.253:35101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 02:32:13.878602 31186 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.253:35101: Network error: Client connection negotiation failed: client connection to 127.17.96.253:35101: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.253.keytab.
Entry for principal kudu/127.17.96.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.253.keytab.
I20260619 02:32:13.884498 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.253:35101
--webserver_interface=127.17.96.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.253.keytab
--principal=kudu/127.17.96.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:46481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:35157,127.17.96.253:35101,127.17.96.252:35493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:32:13.927901 31186 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } attempt: 1
I20260619 02:32:13.928902 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:13.928524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:35101 (local address 127.0.0.1:47768)
0619 02:32:13.928702 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:13.928778 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.253:35101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260619 02:32:13.929724 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:13.929463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:35101 (local address 127.0.0.1:47776)
0619 02:32:13.929603 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:13.929657 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.253:35101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260619 02:32:13.929869 31186 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.253:35101: Network error: Client connection negotiation failed: client connection to 127.17.96.253:35101: connect: Connection refused (error 111)
I20260619 02:32:13.998036 31186 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } attempt: 2
I20260619 02:32:13.998940 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:13.998610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:35101 (local address 127.0.0.1:47786)
0619 02:32:13.998780 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:13.998851 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.253:35101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260619 02:32:13.999814 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:13.999565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:35101 (local address 127.0.0.1:47796)
0619 02:32:13.999694 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:13.999746 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.253:35101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260619 02:32:13.999534 31193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:13.999920 31193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:13.999967 31186 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.253:35101: Network error: Client connection negotiation failed: client connection to 127.17.96.253:35101: connect: Connection refused (error 111)
W20260619 02:32:14.000073 31193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:14.004410 31193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:32:14.004501 31193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:14.004522 31193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:14.004540 31193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:32:14.004607 31193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:32:14.010375 31193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:46481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:35157,127.17.96.253:35101,127.17.96.252:35493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.253.keytab
--principal=kudu/127.17.96.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.253:35101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.96.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:14.011603 31193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:14.012751 31193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:14.022351 31201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:14.022441 31199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:14.022351 31198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:14.022996 31193 server_base.cc:1034] running on GCE node
I20260619 02:32:14.023666 31193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:14.024977 31193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:14.026177 31193 hybrid_clock.cc:648] HybridClock initialized: now 1781836334026142 us; error 55 us; skew 500 ppm
Jun 19 02:32:14 dist-test-slave-4qw6 krb5kdc[31105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836334, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:14.029335 31193 init.cc:377] Logged in from keytab as kudu/127.17.96.253@KRBTEST.COM (short username kudu)
I20260619 02:32:14.030402 31193 webserver.cc:460] Webserver started at http://127.17.96.253:45747/ using document root <none> and password file <none>
I20260619 02:32:14.031041 31193 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:14.031109 31193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:14.031287 31193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:32:14.033411 31193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-1/data/instance:
uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a"
format_stamp: "Formatted at 2026-06-19 02:32:14 on dist-test-slave-4qw6"
server_key: "90db0efa69c29b88567ce79632d6fe86"
server_key_iv: "e5545453e4ff5d51af1b4bcaa77e0663"
server_key_version: "encryptionkey@0"
I20260619 02:32:14.034027 31193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-1/wal/instance:
uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a"
format_stamp: "Formatted at 2026-06-19 02:32:14 on dist-test-slave-4qw6"
server_key: "90db0efa69c29b88567ce79632d6fe86"
server_key_iv: "e5545453e4ff5d51af1b4bcaa77e0663"
server_key_version: "encryptionkey@0"
I20260619 02:32:14.037982 31193 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 02:32:14.040548 31208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:14.041996 31193 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:32:14.042150 31193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-1/wal
uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a"
format_stamp: "Formatted at 2026-06-19 02:32:14 on dist-test-slave-4qw6"
server_key: "90db0efa69c29b88567ce79632d6fe86"
server_key_iv: "e5545453e4ff5d51af1b4bcaa77e0663"
server_key_version: "encryptionkey@0"
I20260619 02:32:14.042273 31193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:14.062320 31193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:14.065894 31193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:14.066115 31193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:14.073549 31193 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.253:35101
I20260619 02:32:14.073567 31260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.253:35101 every 8 connection(s)
I20260619 02:32:14.074638 31193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-1/data/info.pb
I20260619 02:32:14.077558 31261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:32:14.081812 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 31193
I20260619 02:32:14.081948 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-1/wal/instance
I20260619 02:32:14.081555 31261 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35157 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:32:14.082228 17795 external_mini_cluster.cc:1428] Setting key baf124d043e8b1a27c56cdbc18fcd4ac
WARNING: no policy specified for kudu/127.17.96.252@KRBTEST.COM; defaulting to no policy
I20260619 02:32:14.091174 31186 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.252@KRBTEST.COM" created.
Jun 19 02:32:14 dist-test-slave-4qw6 krb5kdc[31105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836334, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.253@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
Jun 19 02:32:14 dist-test-slave-4qw6 krb5kdc[31105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836333, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for kudu/127.17.96.253@KRBTEST.COM
I20260619 02:32:14.099786 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.083842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40804 (local address 127.17.96.254:35157)
0619 02:32:14.084025 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:14.084030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:14.084976 (+ 946us) server_negotiation.cc:407] Connection header received
0619 02:32:14.086102 (+ 1126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:14.086108 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:14.086185 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:14.086303 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:14.087597 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.088521 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.089725 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.090170 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.090289 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:14.094251 (+ 3962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:14.094257 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:14.094260 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:14.094291 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:14.097010 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.097825 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.097831 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.097836 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.097934 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.098404 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.098409 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.098410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.098639 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:14.098769 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:14.099454 (+ 685us) server_negotiation.cc:299] Negotiation successful
0619 02:32:14.099624 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 02:32:14.101536 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.084284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:35157 (local address 127.0.0.1:40804)
0619 02:32:14.084810 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.084865 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:14.085850 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:14.086509 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:14.086517 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:14.086891 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:14.087408 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:14.087423 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.088669 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.088672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:14.089561 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:14.089570 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.090334 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.090380 (+ 46us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:14.090632 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:14.091297 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:14.091314 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:14.094088 (+ 2774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:14.097152 (+ 3064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:14.097160 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:14.097179 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:14.097623 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:14.098058 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:14.098062 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:14.098063 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:14.098283 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:14.098767 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:14.098776 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:14.099092 (+ 316us) client_negotiation.cc:769] Sending connection context
0619 02:32:14.100028 (+ 936us) client_negotiation.cc:240] Negotiation successful
0619 02:32:14.100455 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":172,"threads_started":1}
I20260619 02:32:14.102936 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.091694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:35101 (local address 127.0.0.1:47810)
0619 02:32:14.091858 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.091873 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:14.091959 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:14.092684 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:14.092687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:14.092906 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:14.093053 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:14.093061 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.094309 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.094316 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:14.095169 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:14.095179 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.095833 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.095837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:14.095953 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:14.096434 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:14.096467 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:14.098064 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:14.100608 (+ 2544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:14.100613 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:14.100618 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:14.100914 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:14.101292 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:14.101297 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:14.101302 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:14.101411 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:14.102379 (+ 968us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:14.102382 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:14.102522 (+ 140us) client_negotiation.cc:769] Sending connection context
0619 02:32:14.102648 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 02:32:14.102806 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260619 02:32:14.103379 31261 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:32:14.103713 31265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.091856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47810 (local address 127.17.96.253:35101)
0619 02:32:14.092224 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:14.092238 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:14.092293 (+ 55us) server_negotiation.cc:407] Connection header received
0619 02:32:14.092347 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:14.092352 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:14.092531 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:14.092703 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:14.093193 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.094111 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.095307 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.095709 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.095831 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:14.098236 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:14.098250 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:14.098270 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:14.098304 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:14.100468 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.101053 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.101060 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.101062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.101167 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.101533 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.101537 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.101538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.101961 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:14.102420 (+ 459us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:14.103355 (+ 935us) server_negotiation.cc:299] Negotiation successful
0619 02:32:14.103547 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":161,"threads_started":1}
I20260619 02:32:14.106266 31186 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:32:14.107177 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.106860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35493 (local address 127.0.0.1:49320)
0619 02:32:14.107012 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.107094 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260619 02:32:14.108139 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.107786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35493 (local address 127.0.0.1:49332)
0619 02:32:14.107997 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.108056 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260619 02:32:14.108537 31186 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.252:35493: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
Jun 19 02:32:14 dist-test-slave-4qw6 krb5kdc[31105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836334, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.253@KRBTEST.COM for kudu/127.17.96.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.252.keytab.
Entry for principal kudu/127.17.96.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.252.keytab.
I20260619 02:32:14.113673 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.252:35493
--webserver_interface=127.17.96.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.252.keytab
--principal=kudu/127.17.96.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:46481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:35157,127.17.96.253:35101,127.17.96.252:35493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:32:14.114148 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.104013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:35101 (local address 127.0.0.1:47816)
0619 02:32:14.104152 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.104168 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:14.104260 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:14.104755 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:14.104758 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:14.104989 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:14.105168 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:14.105180 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.106272 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.106281 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:14.107115 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:14.107135 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.107852 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.107857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:14.107982 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:14.108573 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:14.108587 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:14.110052 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:14.112083 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:14.112088 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:14.112090 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:14.112345 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:14.112681 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:14.112687 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:14.112690 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:14.112781 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:14.113310 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:14.113317 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:14.113454 (+ 137us) client_negotiation.cc:769] Sending connection context
0619 02:32:14.113597 (+ 143us) client_negotiation.cc:240] Negotiation successful
0619 02:32:14.113865 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260619 02:32:14.114612 31265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.104288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47816 (local address 127.17.96.253:35101)
0619 02:32:14.104427 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:14.104433 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:14.104449 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:32:14.104486 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:14.104488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:14.104536 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:14.104623 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:14.105331 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.106134 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.107250 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.107705 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.107847 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:14.110248 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:14.110252 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:14.110254 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:14.110277 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:14.111927 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.112465 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.112469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.112472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.112579 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.112907 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.112949 (+ 42us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.112953 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.113198 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:14.113288 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:14.113659 (+ 371us) server_negotiation.cc:299] Negotiation successful
0619 02:32:14.114391 (+ 732us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":597,"server-negotiator.queue_time_us":57}
I20260619 02:32:14.115473 31261 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:32:14.116331 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.116023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35493 (local address 127.0.0.1:49346)
0619 02:32:14.116170 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.116268 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260619 02:32:14.116914 31209 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
I20260619 02:32:14.117899 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.117631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35493 (local address 127.0.0.1:49358)
0619 02:32:14.117777 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.117834 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260619 02:32:14.118085 31261 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.252:35493: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
I20260619 02:32:14.139773 31186 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } attempt: 1
I20260619 02:32:14.140816 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.140445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35493 (local address 127.0.0.1:49366)
0619 02:32:14.140652 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.140724 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260619 02:32:14.141681 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.141407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35493 (local address 127.0.0.1:49380)
0619 02:32:14.141555 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.141616 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260619 02:32:14.141855 31186 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.252:35493: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
I20260619 02:32:14.167374 31261 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } attempt: 1
I20260619 02:32:14.168385 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.168057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35493 (local address 127.0.0.1:49396)
0619 02:32:14.168240 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.168303 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260619 02:32:14.169185 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.168949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35493 (local address 127.0.0.1:49408)
0619 02:32:14.169077 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.169126 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260619 02:32:14.169322 31261 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.252:35493: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
I20260619 02:32:14.217056 31186 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } attempt: 2
I20260619 02:32:14.218189 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.217818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35493 (local address 127.0.0.1:49424)
0619 02:32:14.218013 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.218091 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260619 02:32:14.218953 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.218723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35493 (local address 127.0.0.1:49436)
0619 02:32:14.218839 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.218895 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 02:32:14.219092 31186 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.252:35493: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
W20260619 02:32:14.232091 31267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:14.232462 31267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:14.232556 31267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:14.236862 31267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:32:14.236980 31267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:14.237015 31267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:14.237053 31267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:32:14.237103 31267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:32:14.237514 31261 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } attempt: 2
I20260619 02:32:14.238399 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.238097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35493 (local address 127.0.0.1:49442)
0619 02:32:14.238250 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.238320 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260619 02:32:14.239146 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.238923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35493 (local address 127.0.0.1:49456)
0619 02:32:14.239037 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.239090 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260619 02:32:14.239274 31261 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.252:35493: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
I20260619 02:32:14.243265 31267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:46481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:35157,127.17.96.253:35101,127.17.96.252:35493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.252.keytab
--principal=kudu/127.17.96.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.252:35493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.96.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:14.244457 31267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:14.245901 31267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:14.255439 31273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:14.255400 31274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:14.255443 31276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:14.255664 31267 server_base.cc:1034] running on GCE node
I20260619 02:32:14.256381 31267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:14.257591 31267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:14.258780 31267 hybrid_clock.cc:648] HybridClock initialized: now 1781836334258769 us; error 31 us; skew 500 ppm
Jun 19 02:32:14 dist-test-slave-4qw6 krb5kdc[31105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836334, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:14.264304 31267 init.cc:377] Logged in from keytab as kudu/127.17.96.252@KRBTEST.COM (short username kudu)
I20260619 02:32:14.265333 31267 webserver.cc:460] Webserver started at http://127.17.96.252:40113/ using document root <none> and password file <none>
I20260619 02:32:14.266034 31267 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:14.266137 31267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:14.266391 31267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:32:14.268628 31267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-2/data/instance:
uuid: "b80fa341c2d34d409f057826a751f95a"
format_stamp: "Formatted at 2026-06-19 02:32:14 on dist-test-slave-4qw6"
server_key: "90bdadb4b1063bf7fd37796850d76e65"
server_key_iv: "6ed26ee3539be23ea3e169ee71f768d8"
server_key_version: "encryptionkey@0"
I20260619 02:32:14.269249 31267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-2/wal/instance:
uuid: "b80fa341c2d34d409f057826a751f95a"
format_stamp: "Formatted at 2026-06-19 02:32:14 on dist-test-slave-4qw6"
server_key: "90bdadb4b1063bf7fd37796850d76e65"
server_key_iv: "6ed26ee3539be23ea3e169ee71f768d8"
server_key_version: "encryptionkey@0"
I20260619 02:32:14.273370 31267 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260619 02:32:14.276259 31283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:14.277671 31267 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:32:14.277872 31267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-2/wal
uuid: "b80fa341c2d34d409f057826a751f95a"
format_stamp: "Formatted at 2026-06-19 02:32:14 on dist-test-slave-4qw6"
server_key: "90bdadb4b1063bf7fd37796850d76e65"
server_key_iv: "6ed26ee3539be23ea3e169ee71f768d8"
server_key_version: "encryptionkey@0"
I20260619 02:32:14.278014 31267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:14.318249 31186 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } attempt: 3
I20260619 02:32:14.319134 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.318829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35493 (local address 127.0.0.1:49462)
0619 02:32:14.318982 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.319055 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260619 02:32:14.319904 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.319690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35493 (local address 127.0.0.1:49464)
0619 02:32:14.319798 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.319844 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 02:32:14.320063 31186 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.252:35493: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
I20260619 02:32:14.324082 31267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:14.328313 31267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:14.328569 31267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:14.330417 31261 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } attempt: 3
I20260619 02:32:14.331382 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.330926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35493 (local address 127.0.0.1:49470)
0619 02:32:14.331176 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.331250 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260619 02:32:14.332358 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.332038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35493 (local address 127.0.0.1:49484)
0619 02:32:14.332230 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.332283 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
W20260619 02:32:14.332542 31261 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.252:35493: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35493: connect: Connection refused (error 111)
I20260619 02:32:14.338831 31267 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.252:35493
I20260619 02:32:14.339047 31335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.252:35493 every 8 connection(s)
I20260619 02:32:14.340343 31267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-2/data/info.pb
I20260619 02:32:14.341984 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 31267
I20260619 02:32:14.342139 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781836162001484-17795-0/minicluster-data/master-2/wal/instance
I20260619 02:32:14.342504 17795 external_mini_cluster.cc:1428] Setting key ba97879e9b2c11ddd71d53427afd444f
I20260619 02:32:14.346096 31336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:32:14.352262 31336 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35157 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:32:14 dist-test-slave-4qw6 krb5kdc[31105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.252@KRBTEST.COM
I20260619 02:32:14.361109 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.344131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49500 (local address 127.17.96.252:35493)
0619 02:32:14.344704 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:14.344728 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:14.344779 (+ 51us) server_negotiation.cc:407] Connection header received
0619 02:32:14.345901 (+ 1122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:14.345932 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:14.346450 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:14.346918 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:14.348034 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.349310 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.350298 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.351175 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.351924 (+ 749us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:14.355067 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:14.355077 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:14.355098 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:14.355139 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:14.357549 (+ 2410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.358129 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.358136 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.358142 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.358311 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.358659 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.358665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.358667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.359215 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:14.359456 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:14.359806 (+ 350us) server_negotiation.cc:299] Negotiation successful
0619 02:32:14.360258 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":142,"threads_started":1}
Jun 19 02:32:14 dist-test-slave-4qw6 krb5kdc[31105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836334, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.252@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:14.368572 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.354697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40812 (local address 127.17.96.254:35157)
0619 02:32:14.354844 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:14.354852 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:14.355598 (+ 746us) server_negotiation.cc:407] Connection header received
0619 02:32:14.355692 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:14.355696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:14.355756 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:14.355844 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:14.356999 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.357823 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.359002 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.359341 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.359447 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:14.364094 (+ 4647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:14.364101 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:14.364106 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:14.364147 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:14.366092 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.366792 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.366797 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.366801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.366904 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.367271 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.367275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.367277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.367552 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:14.367638 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:14.368247 (+ 609us) server_negotiation.cc:299] Negotiation successful
0619 02:32:14.368395 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 02:32:14.369020 31340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.354665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:35157 (local address 127.0.0.1:40812)
0619 02:32:14.355438 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.355497 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:14.355616 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:14.356014 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:14.356022 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:14.356637 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:14.356834 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:14.356845 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.357951 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.357957 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:14.358860 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:14.358872 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.359582 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.359584 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:14.359741 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:14.361687 (+ 1946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:14.361708 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:14.363899 (+ 2191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:14.366250 (+ 2351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:14.366263 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:14.366271 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:14.366637 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:14.367039 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:14.367044 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:14.367049 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:14.367149 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:14.367754 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:14.367764 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:14.367930 (+ 166us) client_negotiation.cc:769] Sending connection context
0619 02:32:14.368121 (+ 191us) client_negotiation.cc:240] Negotiation successful
0619 02:32:14.368398 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":171,"threads_started":1}
I20260619 02:32:14.370590 31336 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:32:14 dist-test-slave-4qw6 krb5kdc[31105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:14.376695 31341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.364637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40814 (local address 127.17.96.254:35157)
0619 02:32:14.365053 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:14.365066 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:14.365081 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:32:14.365230 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:14.365237 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:14.365304 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:14.365392 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:14.366357 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.367420 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.368430 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.368791 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.368897 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:14.372199 (+ 3302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:14.372206 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:14.372209 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:14.372243 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:14.374261 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.374785 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.374789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.374791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.374884 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.375206 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.375212 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.375217 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.375502 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:14.375789 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:14.376242 (+ 453us) server_negotiation.cc:299] Negotiation successful
0619 02:32:14.376407 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":130,"threads_started":1}
Jun 19 02:32:14 dist-test-slave-4qw6 krb5kdc[31105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836334, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.252@KRBTEST.COM for kudu/127.17.96.253@KRBTEST.COM
Jun 19 02:32:14 dist-test-slave-4qw6 krb5kdc[31105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.253@KRBTEST.COM
I20260619 02:32:14.385576 31340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.371219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:35101 (local address 127.0.0.1:47832)
0619 02:32:14.371455 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.371469 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:14.371626 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:14.372078 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:14.372084 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:14.372278 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:14.372459 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:14.372467 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.373643 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.373646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:14.374547 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:14.374562 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.375349 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.375354 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:14.375483 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:14.375979 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:14.375995 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:14.377558 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:14.383871 (+ 6313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:14.383875 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:14.383876 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:14.384101 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:14.384437 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:14.384439 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:14.384440 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:14.384519 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:14.384991 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:14.384997 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:14.385130 (+ 133us) client_negotiation.cc:769] Sending connection context
0619 02:32:14.385260 (+ 130us) client_negotiation.cc:240] Negotiation successful
0619 02:32:14.385436 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260619 02:32:14.385659 31265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.371341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47832 (local address 127.17.96.253:35101)
0619 02:32:14.371471 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:14.371476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:14.371521 (+ 45us) server_negotiation.cc:407] Connection header received
0619 02:32:14.371767 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:14.371773 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:14.371843 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:14.371933 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:14.372619 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.373465 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.374727 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.375091 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.375179 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:14.377793 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:14.377798 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:14.377800 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:14.377833 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:14.383735 (+ 5902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.384224 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.384229 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.384234 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.384336 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.384613 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.384617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.384619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.384879 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:14.384985 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:14.385315 (+ 330us) server_negotiation.cc:299] Negotiation successful
0619 02:32:14.385470 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 02:32:14.386718 31336 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:32:14.388990 31342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.378934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47842 (local address 127.17.96.253:35101)
0619 02:32:14.379361 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:14.379368 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:14.379380 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:32:14.379412 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:14.379414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:14.379451 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:14.379531 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:14.380103 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.380656 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.381373 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.381681 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.381841 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:14.385439 (+ 3598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:14.385443 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:14.385447 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:14.385469 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:14.387142 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.387654 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.387666 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.387670 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.387752 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.388036 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.388041 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.388042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.388299 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:14.388544 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:14.388671 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 02:32:14.388797 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":140,"threads_started":1}
Jun 19 02:32:14 dist-test-slave-4qw6 krb5kdc[31105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836334, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.252@KRBTEST.COM for kudu/127.17.96.252@KRBTEST.COM
I20260619 02:32:14.392794 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:32:14.396371 31340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.387309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35493 (local address 127.0.0.1:49508)
0619 02:32:14.387430 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.387443 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:14.387531 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:14.387865 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:14.387868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:14.388026 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:14.388096 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:14.388101 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.389535 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.389539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:14.390230 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:14.390241 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.390714 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.390718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:14.390795 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:14.391320 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:14.391335 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:14.392893 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:14.394748 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:14.394755 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:14.394757 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:14.394954 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:14.395266 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:14.395268 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:14.395270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:14.395357 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:14.395831 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:14.395834 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:14.395917 (+ 83us) client_negotiation.cc:769] Sending connection context
0619 02:32:14.396002 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 02:32:14.396158 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 02:32:14.397065 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.387573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49508 (local address 127.17.96.252:35493)
0619 02:32:14.387687 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:14.387690 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:14.387699 (+ 9us) server_negotiation.cc:407] Connection header received
0619 02:32:14.387724 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:14.387726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:14.387757 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:14.387844 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:14.388223 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.389416 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.390356 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.390603 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.390908 (+ 305us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:14.393028 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:14.393032 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:14.393034 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:14.393065 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:14.394627 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.395053 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.395058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.395060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.395165 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.395465 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.395470 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.395471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.395733 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:14.396688 (+ 955us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:14.396763 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 02:32:14.396890 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 02:32:14.400990 31336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a: Bootstrap starting.
I20260619 02:32:14.404398 31336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a: Neither blocks nor log segments found. Creating new log.
I20260619 02:32:14.405426 31336 log.cc:824] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:32:14.405639 31341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.395258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40820 (local address 127.17.96.254:35157)
0619 02:32:14.395423 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:14.395429 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:14.395764 (+ 335us) server_negotiation.cc:407] Connection header received
0619 02:32:14.395821 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:14.395824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:14.395880 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:14.395951 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:14.397056 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.397902 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.399036 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.399433 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.399540 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:14.401391 (+ 1851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:14.401394 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:14.401397 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:14.401429 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:14.403359 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.403868 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.403872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.403874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.403970 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.404314 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.404318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.404321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.404567 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:14.404716 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:14.405347 (+ 631us) server_negotiation.cc:299] Negotiation successful
0619 02:32:14.405475 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 02:32:14.407995 31336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a: No bootstrap required, opened a new log
I20260619 02:32:14.408339 31342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.395726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47844 (local address 127.17.96.253:35101)
0619 02:32:14.395861 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:14.395868 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:14.396361 (+ 493us) server_negotiation.cc:407] Connection header received
0619 02:32:14.396885 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:14.396889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:14.396943 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:14.397001 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:14.398551 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.399378 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.400934 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.401301 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.401391 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:14.404315 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:14.404321 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:14.404325 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:14.404361 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:14.406236 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.406834 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.406841 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.406844 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.406944 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.407277 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.407280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.407282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.407478 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:14.407626 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:14.407881 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 02:32:14.408016 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 02:32:14.408711 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.396055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49520 (local address 127.17.96.252:35493)
0619 02:32:14.396443 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:14.396454 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:14.396841 (+ 387us) server_negotiation.cc:407] Connection header received
0619 02:32:14.396880 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:14.396882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:14.396928 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:14.397004 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:14.398536 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.399401 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.400644 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.401008 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.401091 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:14.403210 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:14.403216 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:14.403221 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:14.403254 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:14.406392 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.406895 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.406901 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.406904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.406990 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.407338 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.407342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.407343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.407586 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:14.407725 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:14.408383 (+ 658us) server_negotiation.cc:299] Negotiation successful
0619 02:32:14.408531 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":127,"threads_started":1}
I20260619 02:32:14.411972 31336 raft_consensus.cc:348] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "597071c01b4f4ecf9c1a3faa21c48495" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35157 } } peers { permanent_uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } } peers { permanent_uuid: "b80fa341c2d34d409f057826a751f95a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } }
I20260619 02:32:14.412202 31336 raft_consensus.cc:374] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:32:14.412269 31336 raft_consensus.cc:729] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80fa341c2d34d409f057826a751f95a, State: Initialized, Role: FOLLOWER
I20260619 02:32:14.412771 31336 consensus_queue.cc:260] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "597071c01b4f4ecf9c1a3faa21c48495" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35157 } } peers { permanent_uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } } peers { permanent_uuid: "b80fa341c2d34d409f057826a751f95a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } }
I20260619 02:32:14.413302 31352 sys_catalog.cc:455] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "597071c01b4f4ecf9c1a3faa21c48495" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35157 } } peers { permanent_uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } } peers { permanent_uuid: "b80fa341c2d34d409f057826a751f95a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } } }
I20260619 02:32:14.413463 31352 sys_catalog.cc:458] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:32:14.414222 31336 sys_catalog.cc:564] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [sys.catalog]: configured and running, proceeding with master startup.
W20260619 02:32:14.420697 31363 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:32:14.420792 31363 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:32:14.417822 31267 master_runner.cc:186] Error getting master registration for 127.17.96.254:35157: OK, instance_id { permanent_uuid: "597071c01b4f4ecf9c1a3faa21c48495" instance_seqno: 1781836333844655 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:32:14.421169 31267 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:32:14.475737 31261 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } attempt: 4
Jun 19 02:32:14 dist-test-slave-4qw6 krb5kdc[31105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836334, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.253@KRBTEST.COM for kudu/127.17.96.252@KRBTEST.COM
I20260619 02:32:14.484243 31186 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } attempt: 4
I20260619 02:32:14.484939 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.476347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35493 (local address 127.0.0.1:49534)
0619 02:32:14.476499 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.476519 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:14.476624 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:14.476916 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:14.476919 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:14.477132 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:14.477221 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:14.477227 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.478119 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.478124 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:14.478662 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:14.478670 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.479163 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.479167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:14.479261 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:14.479807 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:14.479820 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:14.481275 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:14.483290 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:14.483292 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:14.483294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:14.483493 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:14.483796 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:14.483799 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:14.483801 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:14.483898 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:14.484377 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:14.484381 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:14.484528 (+ 147us) client_negotiation.cc:769] Sending connection context
0619 02:32:14.484616 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 02:32:14.484766 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 02:32:14.485733 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.476441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49534 (local address 127.17.96.252:35493)
0619 02:32:14.476588 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:14.476592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:14.476607 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:32:14.476753 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:14.476756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:14.476807 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:14.476899 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:14.477352 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.478000 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.478787 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.479035 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.479159 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:14.481422 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:14.481426 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:14.481431 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:14.481463 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:14.483185 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.483600 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.483604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.483606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.483696 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.483995 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.483998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.484000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.484247 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:14.485389 (+ 1142us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:14.485465 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 02:32:14.485589 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 02:32:14.488988 31261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a: Bootstrap starting.
Jun 19 02:32:14 dist-test-slave-4qw6 krb5kdc[31105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836333, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for kudu/127.17.96.252@KRBTEST.COM
I20260619 02:32:14.492005 31261 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a: Neither blocks nor log segments found. Creating new log.
I20260619 02:32:14.492944 31261 log.cc:824] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:32:14.493608 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.484944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35493 (local address 127.0.0.1:49550)
0619 02:32:14.485145 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:14.485161 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:14.485245 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:14.485877 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:14.485881 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:14.486083 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:14.486202 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:14.486209 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.486949 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.486956 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:14.487565 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:14.487575 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.488064 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.488068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:14.488154 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:14.488681 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:14.488694 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:14.490035 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:14.491839 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:14.491844 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:14.491846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:14.492103 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:14.492422 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:14.492425 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:14.492427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:14.492521 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:14.492924 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:14.492927 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:14.493117 (+ 190us) client_negotiation.cc:769] Sending connection context
0619 02:32:14.493243 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 02:32:14.493418 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260619 02:32:14.494240 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:14.485435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49550 (local address 127.17.96.252:35493)
0619 02:32:14.485561 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:14.485565 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:14.485578 (+ 13us) server_negotiation.cc:407] Connection header received
0619 02:32:14.485614 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:14.485619 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:14.485666 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:14.485737 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:14.486336 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.486833 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.487690 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:14.487941 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:14.488066 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:14.490179 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:14.490183 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:14.490186 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:14.490218 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:14.491717 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.492238 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.492240 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.492242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.492317 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:14.492621 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:14.492624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:14.492626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:14.492824 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:14.492932 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:14.493933 (+ 1001us) server_negotiation.cc:299] Negotiation successful
0619 02:32:14.494066 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 02:32:14.495543 31261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a: No bootstrap required, opened a new log
I20260619 02:32:14.496868 31186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495: Bootstrap starting.
I20260619 02:32:14.499276 31186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495: Neither blocks nor log segments found. Creating new log.
I20260619 02:32:14.499689 31261 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "597071c01b4f4ecf9c1a3faa21c48495" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35157 } } peers { permanent_uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } } peers { permanent_uuid: "b80fa341c2d34d409f057826a751f95a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } }
I20260619 02:32:14.499922 31261 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:32:14.499974 31261 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1fba1cd4eb4bf0b0abdf4d9a84213a, State: Initialized, Role: FOLLOWER
I20260619 02:32:14.500515 31261 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "597071c01b4f4ecf9c1a3faa21c48495" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35157 } } peers { permanent_uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } } peers { permanent_uuid: "b80fa341c2d34d409f057826a751f95a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } }
I20260619 02:32:14.501163 31186 log.cc:824] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495: Log is configured to *not* fsync() on all Append() calls
I20260619 02:32:14.501181 31366 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "597071c01b4f4ecf9c1a3faa21c48495" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35157 } } peers { permanent_uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } } peers { permanent_uuid: "b80fa341c2d34d409f057826a751f95a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } } }
I20260619 02:32:14.501415 31366 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:32:14.501964 31261 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:32:14.503417 31186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495: No bootstrap required, opened a new log
W20260619 02:32:14.507524 31378 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:32:14.507688 31378 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:32:14.507684 31186 raft_consensus.cc:348] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "597071c01b4f4ecf9c1a3faa21c48495" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35157 } } peers { permanent_uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } } peers { permanent_uuid: "b80fa341c2d34d409f057826a751f95a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } }
I20260619 02:32:14.507927 31186 raft_consensus.cc:374] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:32:14.507989 31186 raft_consensus.cc:729] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 597071c01b4f4ecf9c1a3faa21c48495, State: Initialized, Role: FOLLOWER
I20260619 02:32:14.508569 31186 consensus_queue.cc:260] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "597071c01b4f4ecf9c1a3faa21c48495" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35157 } } peers { permanent_uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } } peers { permanent_uuid: "b80fa341c2d34d409f057826a751f95a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } }
I20260619 02:32:14.505694 31193 master_runner.cc:186] Error getting master registration for 127.17.96.254:35157: OK, instance_id { permanent_uuid: "597071c01b4f4ecf9c1a3faa21c48495" instance_seqno: 1781836333844655 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:32:14.509065 31193 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:32:14.509320 31380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "597071c01b4f4ecf9c1a3faa21c48495" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35157 } } peers { permanent_uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } } peers { permanent_uuid: "b80fa341c2d34d409f057826a751f95a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } } }
I20260619 02:32:14.509454 31380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:32:14.510052 31186 sys_catalog.cc:564] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 02:32:14.515857 31391 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:32:14.515967 31391 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:32:14.521967 31118 master_runner.cc:263] No leader master found from master 597071c01b4f4ecf9c1a3faa21c48495
I20260619 02:32:14.522089 31118 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260619 02:32:14.567901 31214 tablet.cc:2369] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:32:14.580904 31289 tablet.cc:2369] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:32:14.590952 31139 tablet.cc:2369] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:32:14.732750 31352 raft_consensus.cc:482] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:32:14.732961 31352 raft_consensus.cc:504] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "597071c01b4f4ecf9c1a3faa21c48495" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35157 } } peers { permanent_uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } } peers { permanent_uuid: "b80fa341c2d34d409f057826a751f95a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } }
I20260619 02:32:14.733932 31352 leader_election.cc:288] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 597071c01b4f4ecf9c1a3faa21c48495 (127.17.96.254:35157), fc1fba1cd4eb4bf0b0abdf4d9a84213a (127.17.96.253:35101)
I20260619 02:32:14.734747 31235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b80fa341c2d34d409f057826a751f95a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a" is_pre_election: true
I20260619 02:32:14.734767 31160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b80fa341c2d34d409f057826a751f95a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "597071c01b4f4ecf9c1a3faa21c48495" is_pre_election: true
I20260619 02:32:14.735111 31235 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80fa341c2d34d409f057826a751f95a in term 0.
I20260619 02:32:14.735111 31160 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80fa341c2d34d409f057826a751f95a in term 0.
I20260619 02:32:14.735656 31284 leader_election.cc:302] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597071c01b4f4ecf9c1a3faa21c48495, b80fa341c2d34d409f057826a751f95a; no voters:
I20260619 02:32:14.735942 31352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:32:14.736055 31352 raft_consensus.cc:482] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:32:14.736135 31352 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:32:14.737296 31352 raft_consensus.cc:504] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "597071c01b4f4ecf9c1a3faa21c48495" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35157 } } peers { permanent_uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } } peers { permanent_uuid: "b80fa341c2d34d409f057826a751f95a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } }
I20260619 02:32:14.737814 31352 leader_election.cc:288] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [CANDIDATE]: Term 1 election: Requested vote from peers 597071c01b4f4ecf9c1a3faa21c48495 (127.17.96.254:35157), fc1fba1cd4eb4bf0b0abdf4d9a84213a (127.17.96.253:35101)
I20260619 02:32:14.738238 31160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b80fa341c2d34d409f057826a751f95a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "597071c01b4f4ecf9c1a3faa21c48495"
I20260619 02:32:14.738312 31235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b80fa341c2d34d409f057826a751f95a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a"
I20260619 02:32:14.738404 31160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:32:14.738487 31235 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:32:14.739814 31160 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80fa341c2d34d409f057826a751f95a in term 1.
I20260619 02:32:14.739974 31235 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80fa341c2d34d409f057826a751f95a in term 1.
I20260619 02:32:14.740238 31284 leader_election.cc:302] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597071c01b4f4ecf9c1a3faa21c48495, b80fa341c2d34d409f057826a751f95a; no voters:
I20260619 02:32:14.741178 31352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:32:14.741431 31352 raft_consensus.cc:686] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [term 1 LEADER]: Becoming Leader. State: Replica: b80fa341c2d34d409f057826a751f95a, State: Running, Role: LEADER
I20260619 02:32:14.741854 31352 consensus_queue.cc:237] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "597071c01b4f4ecf9c1a3faa21c48495" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35157 } } peers { permanent_uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } } peers { permanent_uuid: "b80fa341c2d34d409f057826a751f95a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } }
I20260619 02:32:14.743884 31393 sys_catalog.cc:455] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b80fa341c2d34d409f057826a751f95a. Latest consensus state: current_term: 1 leader_uuid: "b80fa341c2d34d409f057826a751f95a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "597071c01b4f4ecf9c1a3faa21c48495" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35157 } } peers { permanent_uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } } peers { permanent_uuid: "b80fa341c2d34d409f057826a751f95a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } } }
I20260619 02:32:14.744016 31393 sys_catalog.cc:458] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [sys.catalog]: This master's current role is: LEADER
I20260619 02:32:14.744355 31395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:32:14.745256 31395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:32:14.750517 31235 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a [term 1 FOLLOWER]: Refusing update from remote peer b80fa341c2d34d409f057826a751f95a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:32:14.750699 31160 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495 [term 1 FOLLOWER]: Refusing update from remote peer b80fa341c2d34d409f057826a751f95a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:32:14.751420 31393 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:32:14.751675 31352 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [LEADER]: Connected to new peer: Peer: permanent_uuid: "597071c01b4f4ecf9c1a3faa21c48495" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:32:14.756490 31366 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b80fa341c2d34d409f057826a751f95a. Latest consensus state: current_term: 1 leader_uuid: "b80fa341c2d34d409f057826a751f95a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "597071c01b4f4ecf9c1a3faa21c48495" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35157 } } peers { permanent_uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } } peers { permanent_uuid: "b80fa341c2d34d409f057826a751f95a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } } }
I20260619 02:32:14.756518 31380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b80fa341c2d34d409f057826a751f95a. Latest consensus state: current_term: 1 leader_uuid: "b80fa341c2d34d409f057826a751f95a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "597071c01b4f4ecf9c1a3faa21c48495" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35157 } } peers { permanent_uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } } peers { permanent_uuid: "b80fa341c2d34d409f057826a751f95a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } } }
I20260619 02:32:14.756670 31366 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:32:14.756675 31380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:32:14.760412 31380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b80fa341c2d34d409f057826a751f95a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "597071c01b4f4ecf9c1a3faa21c48495" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35157 } } peers { permanent_uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } } peers { permanent_uuid: "b80fa341c2d34d409f057826a751f95a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } } }
I20260619 02:32:14.760594 31380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:32:14.760537 31366 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b80fa341c2d34d409f057826a751f95a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "597071c01b4f4ecf9c1a3faa21c48495" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35157 } } peers { permanent_uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } } peers { permanent_uuid: "b80fa341c2d34d409f057826a751f95a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } } }
I20260619 02:32:14.760653 31366 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:32:14.761801 31393 sys_catalog.cc:455] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b80fa341c2d34d409f057826a751f95a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "597071c01b4f4ecf9c1a3faa21c48495" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35157 } } peers { permanent_uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } } peers { permanent_uuid: "b80fa341c2d34d409f057826a751f95a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } } }
I20260619 02:32:14.761955 31393 sys_catalog.cc:458] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [sys.catalog]: This master's current role is: LEADER
I20260619 02:32:14.762113 31393 sys_catalog.cc:455] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b80fa341c2d34d409f057826a751f95a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "597071c01b4f4ecf9c1a3faa21c48495" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 35157 } } peers { permanent_uuid: "fc1fba1cd4eb4bf0b0abdf4d9a84213a" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 35101 } } peers { permanent_uuid: "b80fa341c2d34d409f057826a751f95a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35493 } } }
I20260619 02:32:14.762210 31393 sys_catalog.cc:458] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a [sys.catalog]: This master's current role is: LEADER
I20260619 02:32:14.762871 31395 catalog_manager.cc:1347] Generated new cluster ID: 82af9e71d88841afa19884a5d5362f9f
I20260619 02:32:14.762946 31395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:32:14.789534 31395 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:32:14.791145 31395 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:32:14.806993 31395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b80fa341c2d34d409f057826a751f95a: Generated new TSK 0
I20260619 02:32:14.808110 31395 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:32:15.509236 31378 catalog_manager.cc:1259] Loaded cluster ID: 82af9e71d88841afa19884a5d5362f9f
I20260619 02:32:15.509332 31378 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a: loading cluster ID for follower catalog manager: success
I20260619 02:32:15.511310 31378 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a: acquiring CA information for follower catalog manager: success
I20260619 02:32:15.512765 31378 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fc1fba1cd4eb4bf0b0abdf4d9a84213a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:32:15.523885 31391 catalog_manager.cc:1259] Loaded cluster ID: 82af9e71d88841afa19884a5d5362f9f
I20260619 02:32:15.523972 31391 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495: loading cluster ID for follower catalog manager: success
I20260619 02:32:15.525576 31391 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495: acquiring CA information for follower catalog manager: success
I20260619 02:32:15.526767 31391 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 597071c01b4f4ecf9c1a3faa21c48495: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:32:15.880999 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 31118
I20260619 02:32:15.887408 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 31193
I20260619 02:32:15.893630 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 31267
2026-06-19T02:32:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3425 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:32:15 dist-test-slave-4qw6 krb5kdc[31410](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:32:15 dist-test-slave-4qw6 krb5kdc[31410](info): set up 2 sockets
Jun 19 02:32:15 dist-test-slave-4qw6 krb5kdc[31410](info): commencing operation
krb5kdc: starting...
W20260619 02:32:16.930126 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:32:17 dist-test-slave-4qw6 krb5kdc[31410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:32:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:32:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:32:17.058979 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:43905
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:38103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:43905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:32:17.177062 31423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:17.177417 31423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:17.177510 31423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:17.181773 31423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:32:17.181896 31423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:17.181931 31423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:17.181973 31423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:32:17.182008 31423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:32:17.187657 31423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:38103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:43905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:43905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:17.188889 31423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:17.189999 31423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:17.198760 31431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:17.198778 31429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:17.198823 31428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:17.199019 31423 server_base.cc:1034] running on GCE node
I20260619 02:32:17.199736 31423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:17.200913 31423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:17.202102 31423 hybrid_clock.cc:648] HybridClock initialized: now 1781836337202092 us; error 38 us; skew 500 ppm
Jun 19 02:32:17 dist-test-slave-4qw6 krb5kdc[31410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836337, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:17.204972 31423 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:32:17.205978 31423 webserver.cc:460] Webserver started at http://127.17.96.254:46437/ using document root <none> and password file <none>
I20260619 02:32:17.206565 31423 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:17.206660 31423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:17.206900 31423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:32:17.209000 31423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "24714595c69749da87bd463ae4484ce7"
format_stamp: "Formatted at 2026-06-19 02:32:17 on dist-test-slave-4qw6"
server_key: "1c751d44f4723c0f2d80ae01051008d2"
server_key_iv: "d0c4066578cf4b6b8ed29a2f0b3dc924"
server_key_version: "encryptionkey@0"
I20260619 02:32:17.209594 31423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "24714595c69749da87bd463ae4484ce7"
format_stamp: "Formatted at 2026-06-19 02:32:17 on dist-test-slave-4qw6"
server_key: "1c751d44f4723c0f2d80ae01051008d2"
server_key_iv: "d0c4066578cf4b6b8ed29a2f0b3dc924"
server_key_version: "encryptionkey@0"
I20260619 02:32:17.213506 31423 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260619 02:32:17.216296 31438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:17.217585 31423 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:32:17.217738 31423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "24714595c69749da87bd463ae4484ce7"
format_stamp: "Formatted at 2026-06-19 02:32:17 on dist-test-slave-4qw6"
server_key: "1c751d44f4723c0f2d80ae01051008d2"
server_key_iv: "d0c4066578cf4b6b8ed29a2f0b3dc924"
server_key_version: "encryptionkey@0"
I20260619 02:32:17.217926 31423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:17.238710 31423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:17.242278 31423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:17.242502 31423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:17.250347 31423 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:43905
I20260619 02:32:17.250377 31490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:43905 every 8 connection(s)
I20260619 02:32:17.251356 31423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:32:17.254302 31491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:32:17.255954 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 31423
I20260619 02:32:17.256096 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:32:17.256363 17795 external_mini_cluster.cc:1428] Setting key 365f376ede58162507aa842b2f3a22f8
I20260619 02:32:17.261340 31491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24714595c69749da87bd463ae4484ce7: Bootstrap starting.
I20260619 02:32:17.264400 31491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24714595c69749da87bd463ae4484ce7: Neither blocks nor log segments found. Creating new log.
I20260619 02:32:17.265232 31491 log.cc:824] T 00000000000000000000000000000000 P 24714595c69749da87bd463ae4484ce7: Log is configured to *not* fsync() on all Append() calls
Jun 19 02:32:17 dist-test-slave-4qw6 krb5kdc[31410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:17.267756 31491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24714595c69749da87bd463ae4484ce7: No bootstrap required, opened a new log
I20260619 02:32:17.271552 31491 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24714595c69749da87bd463ae4484ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24714595c69749da87bd463ae4484ce7" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 43905 } }
I20260619 02:32:17.271862 31491 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24714595c69749da87bd463ae4484ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:32:17.271945 31491 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24714595c69749da87bd463ae4484ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24714595c69749da87bd463ae4484ce7, State: Initialized, Role: FOLLOWER
I20260619 02:32:17.272466 31491 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24714595c69749da87bd463ae4484ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24714595c69749da87bd463ae4484ce7" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 43905 } }
I20260619 02:32:17.272559 31494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:17.257901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39638 (local address 127.17.96.254:43905)
0619 02:32:17.258360 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:17.258384 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:17.258430 (+ 46us) server_negotiation.cc:407] Connection header received
0619 02:32:17.259299 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:17.259322 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:17.259759 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:17.260155 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:17.260835 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:17.262338 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:17.263470 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:17.264177 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:17.264365 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:17.267346 (+ 2981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:17.267353 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:17.267370 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:17.267404 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:17.269318 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:17.269872 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:17.269878 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:17.269883 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:17.270009 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:17.270317 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:17.270322 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:17.270324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:17.270767 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:17.271000 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:17.271299 (+ 299us) server_negotiation.cc:299] Negotiation successful
0619 02:32:17.271624 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":114,"threads_started":1}
I20260619 02:32:17.272640 31491 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24714595c69749da87bd463ae4484ce7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:32:17.272716 31491 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24714595c69749da87bd463ae4484ce7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:32:17.272845 31491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24714595c69749da87bd463ae4484ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:32:17.274116 31491 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24714595c69749da87bd463ae4484ce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24714595c69749da87bd463ae4484ce7" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 43905 } }
I20260619 02:32:17.274506 31491 leader_election.cc:302] T 00000000000000000000000000000000 P 24714595c69749da87bd463ae4484ce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24714595c69749da87bd463ae4484ce7; no voters:
I20260619 02:32:17.274914 31491 leader_election.cc:288] T 00000000000000000000000000000000 P 24714595c69749da87bd463ae4484ce7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:32:17.274991 31496 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24714595c69749da87bd463ae4484ce7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:32:17.275200 31496 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24714595c69749da87bd463ae4484ce7 [term 1 LEADER]: Becoming Leader. State: Replica: 24714595c69749da87bd463ae4484ce7, State: Running, Role: LEADER
I20260619 02:32:17.275556 31496 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24714595c69749da87bd463ae4484ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24714595c69749da87bd463ae4484ce7" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 43905 } }
I20260619 02:32:17.275754 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:32:17.276259 31491 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24714595c69749da87bd463ae4484ce7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:32:17.277323 31497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24714595c69749da87bd463ae4484ce7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24714595c69749da87bd463ae4484ce7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24714595c69749da87bd463ae4484ce7" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 43905 } } }
I20260619 02:32:17.277446 31497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24714595c69749da87bd463ae4484ce7 [sys.catalog]: This master's current role is: LEADER
I20260619 02:32:17.277851 31498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24714595c69749da87bd463ae4484ce7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24714595c69749da87bd463ae4484ce7. Latest consensus state: current_term: 1 leader_uuid: "24714595c69749da87bd463ae4484ce7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24714595c69749da87bd463ae4484ce7" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 43905 } } }
I20260619 02:32:17.277946 31505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:32:17.278024 31498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24714595c69749da87bd463ae4484ce7 [sys.catalog]: This master's current role is: LEADER
I20260619 02:32:17.281360 31505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:32:17.286793 31505 catalog_manager.cc:1347] Generated new cluster ID: ecc62c97669f44d68c7a2348ddd5f090
I20260619 02:32:17.286882 31505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:32:17.298692 31505 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:32:17.299743 31505 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:32:17.303556 31505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24714595c69749da87bd463ae4484ce7: Generated new TSK 0
I20260619 02:32:17.304284 31505 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:32:17.332377 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 31423
2026-06-19T02:32:17Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1436 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:32:17 dist-test-slave-4qw6 krb5kdc[31518](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:32:17 dist-test-slave-4qw6 krb5kdc[31518](info): set up 2 sockets
Jun 19 02:32:17 dist-test-slave-4qw6 krb5kdc[31518](info): commencing operation
krb5kdc: starting...
W20260619 02:32:18.381969 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:32:18 dist-test-slave-4qw6 krb5kdc[31518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:32:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:32:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:32:18.508929 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:38995
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:36409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:38995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:32:18.625829 31531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:18.626116 31531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:18.626159 31531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:18.630431 31531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:32:18.630499 31531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:18.630520 31531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:18.630538 31531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:32:18.630553 31531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:32:18.636224 31531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:36409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:38995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:38995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:18.637348 31531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:18.638440 31531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:18.647267 31536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:18.647390 31531 server_base.cc:1034] running on GCE node
W20260619 02:32:18.647248 31539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:18.647585 31537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:18.648214 31531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:18.649385 31531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:18.650561 31531 hybrid_clock.cc:648] HybridClock initialized: now 1781836338650519 us; error 58 us; skew 500 ppm
Jun 19 02:32:18 dist-test-slave-4qw6 krb5kdc[31518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836338, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:18.653476 31531 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:32:18.654495 31531 webserver.cc:460] Webserver started at http://127.17.96.254:44783/ using document root <none> and password file <none>
I20260619 02:32:18.655048 31531 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:18.655110 31531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:18.655294 31531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:32:18.657411 31531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "42db01878520488a833834b8996abb50"
format_stamp: "Formatted at 2026-06-19 02:32:18 on dist-test-slave-4qw6"
server_key: "a816f0b3c385997e86c877325a0f0f24"
server_key_iv: "f3c0352120e52bdf37a2dfd81a9e584d"
server_key_version: "encryptionkey@0"
I20260619 02:32:18.658049 31531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "42db01878520488a833834b8996abb50"
format_stamp: "Formatted at 2026-06-19 02:32:18 on dist-test-slave-4qw6"
server_key: "a816f0b3c385997e86c877325a0f0f24"
server_key_iv: "f3c0352120e52bdf37a2dfd81a9e584d"
server_key_version: "encryptionkey@0"
I20260619 02:32:18.661957 31531 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 02:32:18.664500 31546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:18.665849 31531 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 02:32:18.665959 31531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "42db01878520488a833834b8996abb50"
format_stamp: "Formatted at 2026-06-19 02:32:18 on dist-test-slave-4qw6"
server_key: "a816f0b3c385997e86c877325a0f0f24"
server_key_iv: "f3c0352120e52bdf37a2dfd81a9e584d"
server_key_version: "encryptionkey@0"
I20260619 02:32:18.666061 31531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:18.683511 31531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:18.686902 31531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:18.687079 31531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:18.694332 31531 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:38995
I20260619 02:32:18.694643 31598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:38995 every 8 connection(s)
I20260619 02:32:18.695453 31531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:32:18.696103 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 31531
I20260619 02:32:18.696246 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:32:18.696538 17795 external_mini_cluster.cc:1428] Setting key 823cda99e9afb354ace25d187025250e
I20260619 02:32:18.698823 31599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:32:18.706116 31599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50: Bootstrap starting.
Jun 19 02:32:18 dist-test-slave-4qw6 krb5kdc[31518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:18.708580 31599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50: Neither blocks nor log segments found. Creating new log.
I20260619 02:32:18.709389 31599 log.cc:824] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50: Log is configured to *not* fsync() on all Append() calls
I20260619 02:32:18.711872 31599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50: No bootstrap required, opened a new log
I20260619 02:32:18.712353 31602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:18.698148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55406 (local address 127.17.96.254:38995)
0619 02:32:18.698595 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:18.698611 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:18.698647 (+ 36us) server_negotiation.cc:407] Connection header received
0619 02:32:18.699351 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:18.699368 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:18.699714 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:18.700012 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:18.700956 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:18.702058 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:18.703239 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:18.703724 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:18.703871 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:18.707017 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:18.707023 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:18.707038 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:18.707072 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:18.708994 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:18.709497 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:18.709502 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:18.709525 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:18.709667 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:18.710019 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:18.710021 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:18.710023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:18.710555 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:18.710792 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:18.711067 (+ 275us) server_negotiation.cc:299] Negotiation successful
0619 02:32:18.711380 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":142,"threads_started":1}
I20260619 02:32:18.715729 31599 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42db01878520488a833834b8996abb50" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38995 } }
I20260619 02:32:18.715974 31599 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:32:18.716017 31599 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42db01878520488a833834b8996abb50, State: Initialized, Role: FOLLOWER
I20260619 02:32:18.716403 31599 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42db01878520488a833834b8996abb50" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38995 } }
I20260619 02:32:18.716516 31599 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:32:18.716557 31599 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:32:18.716626 31599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:32:18.717598 31599 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42db01878520488a833834b8996abb50" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38995 } }
I20260619 02:32:18.718006 31599 leader_election.cc:302] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42db01878520488a833834b8996abb50; no voters:
I20260619 02:32:18.718391 31599 leader_election.cc:288] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:32:18.718539 31604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:32:18.718801 31604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 1 LEADER]: Becoming Leader. State: Replica: 42db01878520488a833834b8996abb50, State: Running, Role: LEADER
I20260619 02:32:18.719203 31604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42db01878520488a833834b8996abb50" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38995 } }
I20260619 02:32:18.719820 31599 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:32:18.720788 31605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42db01878520488a833834b8996abb50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42db01878520488a833834b8996abb50" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38995 } } }
I20260619 02:32:18.720966 31605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [sys.catalog]: This master's current role is: LEADER
I20260619 02:32:18.720997 31606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42db01878520488a833834b8996abb50. Latest consensus state: current_term: 1 leader_uuid: "42db01878520488a833834b8996abb50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42db01878520488a833834b8996abb50" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38995 } } }
I20260619 02:32:18.721177 31606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [sys.catalog]: This master's current role is: LEADER
I20260619 02:32:18.721388 31613 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:32:18.724725 31613 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:32:18.731308 31613 catalog_manager.cc:1347] Generated new cluster ID: e42653a319c34db7b69e4bbf17ba83c1
I20260619 02:32:18.731410 31613 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:32:18.756697 31613 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:32:18.758193 31613 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:32:18.763777 31613 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50: Generated new TSK 0
I20260619 02:32:18.764624 31613 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.96.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
I20260619 02:32:18.797179 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:38995
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:36409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:32:18.912441 31625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:18.912789 31625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:18.912873 31625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:18.917157 31625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:18.917239 31625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:18.917303 31625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:32:18.923242 31625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:36409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:38995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:18.924386 31625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:18.925424 31625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:18.935568 31633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:18.935598 31630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:18.935561 31631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:18.935890 31625 server_base.cc:1034] running on GCE node
I20260619 02:32:18.936455 31625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:18.937143 31625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:18.938335 31625 hybrid_clock.cc:648] HybridClock initialized: now 1781836338938313 us; error 45 us; skew 500 ppm
Jun 19 02:32:18 dist-test-slave-4qw6 krb5kdc[31518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836338, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:18.941280 31625 init.cc:377] Logged in from keytab as kudu/127.17.96.193@KRBTEST.COM (short username kudu)
I20260619 02:32:18.942296 31625 webserver.cc:460] Webserver started at http://127.17.96.193:38311/ using document root <none> and password file <none>
I20260619 02:32:18.942888 31625 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:18.942981 31625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:18.943218 31625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:32:18.945336 31625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9"
format_stamp: "Formatted at 2026-06-19 02:32:18 on dist-test-slave-4qw6"
server_key: "a7fbbc45af4bd27a50fe135dcd2573ce"
server_key_iv: "09474d6bddf9ff8824b6803bcf149d2d"
server_key_version: "encryptionkey@0"
I20260619 02:32:18.945937 31625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9"
format_stamp: "Formatted at 2026-06-19 02:32:18 on dist-test-slave-4qw6"
server_key: "a7fbbc45af4bd27a50fe135dcd2573ce"
server_key_iv: "09474d6bddf9ff8824b6803bcf149d2d"
server_key_version: "encryptionkey@0"
I20260619 02:32:18.949715 31625 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 02:32:18.952276 31640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:18.953639 31625 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 02:32:18.953812 31625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9"
format_stamp: "Formatted at 2026-06-19 02:32:18 on dist-test-slave-4qw6"
server_key: "a7fbbc45af4bd27a50fe135dcd2573ce"
server_key_iv: "09474d6bddf9ff8824b6803bcf149d2d"
server_key_version: "encryptionkey@0"
I20260619 02:32:18.953956 31625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:18.973007 31625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:18.976400 31625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:18.976622 31625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:18.977249 31625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:32:18.978174 31625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:32:18.978242 31625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:18.978310 31625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:32:18.978356 31625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:18.987577 31625 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:32897
I20260619 02:32:18.987597 31753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:32897 every 8 connection(s)
I20260619 02:32:18.988528 31625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:32:18.994369 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 31625
I20260619 02:32:18.994532 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:32:18.994858 17795 external_mini_cluster.cc:1428] Setting key 8dd1966f8561f8507ad43977e70f59e4
Jun 19 02:32:18 dist-test-slave-4qw6 krb5kdc[31518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836338, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:19.004762 31602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:18.990510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:46059 (local address 127.17.96.254:38995)
0619 02:32:18.990694 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:18.990699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:18.991586 (+ 887us) server_negotiation.cc:407] Connection header received
0619 02:32:18.992956 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:18.992959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:18.993012 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:18.993094 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:18.994389 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:18.995225 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:18.996869 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:18.997302 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:18.997430 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:19.000429 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:19.000437 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:19.000443 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:19.000479 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:19.002364 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:19.002987 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:19.002993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:19.002997 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:19.003117 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:19.003473 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:19.003479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:19.003481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:19.003752 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:19.003870 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:19.004397 (+ 527us) server_negotiation.cc:299] Negotiation successful
0619 02:32:19.004556 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 02:32:19.005577 31756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:18.990916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.193:46059)
0619 02:32:18.991424 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:18.991469 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:18.992745 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:18.993272 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:18.993279 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:18.993679 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:18.994221 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:18.994235 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:18.995390 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:18.995394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:18.996737 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:18.996749 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:18.997458 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:18.997461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:18.997621 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:18.998433 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:18.998461 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:19.000238 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:19.002532 (+ 2294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:19.002538 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:19.002553 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:19.002867 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:19.003221 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:19.003224 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:19.003225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:19.003376 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:19.003866 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:19.003871 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:19.004136 (+ 265us) client_negotiation.cc:769] Sending connection context
0619 02:32:19.004306 (+ 170us) client_negotiation.cc:240] Negotiation successful
0619 02:32:19.004602 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":135,"threads_started":1}
WARNING: no policy specified for kudu/127.17.96.194@KRBTEST.COM; defaulting to no policy
I20260619 02:32:19.007174 31754 heartbeater.cc:347] Connected to a master server at 127.17.96.254:38995
I20260619 02:32:19.007472 31754 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:19.008060 31754 heartbeater.cc:511] Master 127.17.96.254:38995 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.194@KRBTEST.COM" created.
I20260619 02:32:19.009996 31563 ts_manager.cc:194] Registered new tserver with Master: aa0f20ec8cad41f3934b52f1c3ce7bc9 (127.17.96.193:32897)
I20260619 02:32:19.011179 31563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:46059
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
I20260619 02:32:19.025272 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:38995
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:36409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:32:19.138036 31759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:19.138336 31759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:19.138424 31759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:19.142604 31759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:19.142673 31759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:19.142803 31759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:32:19.148427 31759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:36409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:38995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:19.149626 31759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:19.150739 31759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:19.161049 31764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:19.161049 31765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:19.161072 31767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:19.161404 31759 server_base.cc:1034] running on GCE node
I20260619 02:32:19.161913 31759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:19.162603 31759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:19.163797 31759 hybrid_clock.cc:648] HybridClock initialized: now 1781836339163773 us; error 47 us; skew 500 ppm
Jun 19 02:32:19 dist-test-slave-4qw6 krb5kdc[31518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836339, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:19.166751 31759 init.cc:377] Logged in from keytab as kudu/127.17.96.194@KRBTEST.COM (short username kudu)
I20260619 02:32:19.167759 31759 webserver.cc:460] Webserver started at http://127.17.96.194:37521/ using document root <none> and password file <none>
I20260619 02:32:19.168383 31759 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:19.168479 31759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:19.168722 31759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:32:19.171223 31759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "86f29e57a9774074b91d28ad2d342074"
format_stamp: "Formatted at 2026-06-19 02:32:19 on dist-test-slave-4qw6"
server_key: "a86468564d28b96e3c410f9ab79419a6"
server_key_iv: "9dc979305a04cf0518e8a427cf18309f"
server_key_version: "encryptionkey@0"
I20260619 02:32:19.171834 31759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "86f29e57a9774074b91d28ad2d342074"
format_stamp: "Formatted at 2026-06-19 02:32:19 on dist-test-slave-4qw6"
server_key: "a86468564d28b96e3c410f9ab79419a6"
server_key_iv: "9dc979305a04cf0518e8a427cf18309f"
server_key_version: "encryptionkey@0"
I20260619 02:32:19.175873 31759 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:32:19.178576 31774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:19.179910 31759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:32:19.180065 31759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "86f29e57a9774074b91d28ad2d342074"
format_stamp: "Formatted at 2026-06-19 02:32:19 on dist-test-slave-4qw6"
server_key: "a86468564d28b96e3c410f9ab79419a6"
server_key_iv: "9dc979305a04cf0518e8a427cf18309f"
server_key_version: "encryptionkey@0"
I20260619 02:32:19.180202 31759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:19.199601 31759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:19.203253 31759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:19.203472 31759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:19.204104 31759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:32:19.204975 31759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:32:19.205049 31759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:19.205129 31759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:32:19.205181 31759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:19.214874 31759 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:44865
I20260619 02:32:19.214892 31887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:44865 every 8 connection(s)
I20260619 02:32:19.215816 31759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:32:19.222118 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 31759
I20260619 02:32:19.222277 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:32:19.222558 17795 external_mini_cluster.cc:1428] Setting key 824e427c67029344166b25b09dbe338c
Jun 19 02:32:19 dist-test-slave-4qw6 krb5kdc[31518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836339, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:19.232327 31602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:19.217696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:60139 (local address 127.17.96.254:38995)
0619 02:32:19.217886 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:19.217889 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:19.218750 (+ 861us) server_negotiation.cc:407] Connection header received
0619 02:32:19.220292 (+ 1542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:19.220296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:19.220360 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:19.220474 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:19.222037 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.222759 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.224264 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.224478 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.224563 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:19.227859 (+ 3296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:19.227866 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:19.227872 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:19.227908 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:19.229809 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:19.230383 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:19.230390 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:19.230393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:19.230501 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:19.230961 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:19.230966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:19.230968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:19.231254 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:19.231365 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:19.231964 (+ 599us) server_negotiation.cc:299] Negotiation successful
0619 02:32:19.232120 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
WARNING: no policy specified for kudu/127.17.96.195@KRBTEST.COM; defaulting to no policy
I20260619 02:32:19.233116 31890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:19.218120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.194:60139)
0619 02:32:19.218579 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:19.218625 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:19.220060 (+ 1435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:19.220670 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:19.220678 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:19.221216 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:19.221838 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:19.221884 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.222898 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.222902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:19.224136 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:19.224149 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.224658 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.224665 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:19.224915 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:19.225607 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:19.225635 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:19.227659 (+ 2024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:19.229942 (+ 2283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:19.229948 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:19.229962 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:19.230253 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:19.230652 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:19.230655 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:19.230656 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:19.230829 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:19.231392 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:19.231398 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:19.231676 (+ 278us) client_negotiation.cc:769] Sending connection context
0619 02:32:19.231849 (+ 173us) client_negotiation.cc:240] Negotiation successful
0619 02:32:19.232153 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":126,"threads_started":1}
I20260619 02:32:19.234438 31888 heartbeater.cc:347] Connected to a master server at 127.17.96.254:38995
I20260619 02:32:19.234678 31888 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:19.235253 31888 heartbeater.cc:511] Master 127.17.96.254:38995 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.195@KRBTEST.COM" created.
I20260619 02:32:19.236444 31563 ts_manager.cc:194] Registered new tserver with Master: 86f29e57a9774074b91d28ad2d342074 (127.17.96.194:44865)
I20260619 02:32:19.237062 31563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:60139
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
I20260619 02:32:19.250607 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:38995
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:36409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:32:19.365478 31893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:19.365852 31893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:19.365906 31893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:19.370138 31893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:19.370208 31893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:19.370273 31893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:32:19.375847 31893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:36409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:38995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:19.376964 31893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:19.377991 31893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:19.388166 31899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:19.388142 31901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:19.388144 31898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:19.388643 31893 server_base.cc:1034] running on GCE node
I20260619 02:32:19.389094 31893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:19.389855 31893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:19.391048 31893 hybrid_clock.cc:648] HybridClock initialized: now 1781836339391027 us; error 44 us; skew 500 ppm
Jun 19 02:32:19 dist-test-slave-4qw6 krb5kdc[31518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836339, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:19.397063 31893 init.cc:377] Logged in from keytab as kudu/127.17.96.195@KRBTEST.COM (short username kudu)
I20260619 02:32:19.398171 31893 webserver.cc:460] Webserver started at http://127.17.96.195:36531/ using document root <none> and password file <none>
I20260619 02:32:19.398742 31893 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:19.398801 31893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:19.398972 31893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:32:19.401049 31893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a"
format_stamp: "Formatted at 2026-06-19 02:32:19 on dist-test-slave-4qw6"
server_key: "99d101fb6908af95dd44dec776a1e657"
server_key_iv: "6facbd1753831c3e57d6145b7b0212ad"
server_key_version: "encryptionkey@0"
I20260619 02:32:19.401580 31893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a"
format_stamp: "Formatted at 2026-06-19 02:32:19 on dist-test-slave-4qw6"
server_key: "99d101fb6908af95dd44dec776a1e657"
server_key_iv: "6facbd1753831c3e57d6145b7b0212ad"
server_key_version: "encryptionkey@0"
I20260619 02:32:19.405527 31893 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260619 02:32:19.408111 31908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:19.409435 31893 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 02:32:19.409562 31893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a"
format_stamp: "Formatted at 2026-06-19 02:32:19 on dist-test-slave-4qw6"
server_key: "99d101fb6908af95dd44dec776a1e657"
server_key_iv: "6facbd1753831c3e57d6145b7b0212ad"
server_key_version: "encryptionkey@0"
I20260619 02:32:19.409663 31893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:19.421205 31893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:19.424299 31893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:19.424465 31893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:19.425012 31893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:32:19.425896 31893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:32:19.425943 31893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:19.425985 31893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:32:19.426009 31893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:19.435302 31893 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:46419
I20260619 02:32:19.435317 32021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:46419 every 8 connection(s)
I20260619 02:32:19.436290 31893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:32:19.438189 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 31893
I20260619 02:32:19.438328 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:32:19.438629 17795 external_mini_cluster.cc:1428] Setting key b3fb2bd1432285bff76ef4ed5c8bcc7d
Jun 19 02:32:19 dist-test-slave-4qw6 krb5kdc[31518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836339, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:19.450943 31602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:19.438203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:33785 (local address 127.17.96.254:38995)
0619 02:32:19.438338 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:19.438350 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:19.439473 (+ 1123us) server_negotiation.cc:407] Connection header received
0619 02:32:19.441417 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:19.441419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:19.441466 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:19.441560 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:19.442772 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.443330 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.444529 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.444746 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.444828 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:19.447584 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:19.447588 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:19.447591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:19.447621 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:19.448980 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:19.449486 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:19.449489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:19.449490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:19.449570 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:19.449939 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:19.449941 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:19.449942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:19.450133 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:19.450213 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:19.450697 (+ 484us) server_negotiation.cc:299] Negotiation successful
0619 02:32:19.450805 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 02:32:19.452000 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:19.438739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.195:33785)
0619 02:32:19.439298 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:19.439362 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:19.441233 (+ 1871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:19.441705 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:19.441711 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:19.442148 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:19.442646 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:19.442660 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.443441 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.443443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:19.444396 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:19.444411 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.444886 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.444889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:19.445037 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:19.445611 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:19.445646 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:19.447385 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:19.449095 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:19.449101 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:19.449120 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:19.449385 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:19.449651 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:19.449653 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:19.449654 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:19.449849 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:19.450219 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:19.450224 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:19.450475 (+ 251us) client_negotiation.cc:769] Sending connection context
0619 02:32:19.450640 (+ 165us) client_negotiation.cc:240] Negotiation successful
0619 02:32:19.450915 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":163,"threads_started":1}
I20260619 02:32:19.453276 32022 heartbeater.cc:347] Connected to a master server at 127.17.96.254:38995
I20260619 02:32:19.453544 32022 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:19.454139 32022 heartbeater.cc:511] Master 127.17.96.254:38995 requested a full tablet report, sending...
I20260619 02:32:19.455281 31562 ts_manager.cc:194] Registered new tserver with Master: 5ec16fb3eb8a43fd9d0a863f7ce89e7a (127.17.96.195:46419)
I20260619 02:32:19.455942 31562 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:33785
I20260619 02:32:19.465153 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:32:19.474591 31602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:19.467275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55408 (local address 127.17.96.254:38995)
0619 02:32:19.467425 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:19.467430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:19.467582 (+ 152us) server_negotiation.cc:407] Connection header received
0619 02:32:19.467687 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:19.467690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:19.467735 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:19.467797 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:19.468488 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.469019 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.470018 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.470236 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.470309 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:19.471609 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:19.471612 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:19.471615 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:19.471642 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:19.473044 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:19.473408 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:19.473410 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:19.473412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:19.473485 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:19.473714 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:19.473717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:19.473718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:19.473980 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:19.474084 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:19.474284 (+ 200us) server_negotiation.cc:299] Negotiation successful
0619 02:32:19.474379 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 02:32:19.477497 31562 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55408:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:32:19.480633 31562 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:32:19.495687 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:19.490108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33860 (local address 127.17.96.194:44865)
0619 02:32:19.490545 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:19.490557 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:19.490582 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:32:19.490665 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:19.490671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:19.490888 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:19.491045 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:32:19.491411 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.492660 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.494272 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.494849 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.494982 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:19.495163 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:19.495432 (+ 269us) server_negotiation.cc:299] Negotiation successful
0619 02:32:19.495533 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":100,"threads_started":1}
I20260619 02:32:19.495687 32035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:19.489361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47662 (local address 127.17.96.195:46419)
0619 02:32:19.490933 (+ 1572us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:19.490949 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:19.490970 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:32:19.491042 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:19.491047 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:19.491234 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:19.491611 (+ 377us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:32:19.491737 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.492420 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.494092 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.494963 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.495193 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:19.495348 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:19.495407 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 02:32:19.495511 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1469,"thread_start_us":125,"threads_started":1}
I20260619 02:32:19.495679 32034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:19.489557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:32897 (local address 127.0.0.1:37096)
0619 02:32:19.490570 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:19.490582 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:19.490671 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:19.491259 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:19.491262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:19.491283 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:32:19.491417 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:19.491424 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.492813 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.492816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:19.494093 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:19.494104 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.495126 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.495131 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:19.495228 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:19.495251 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 02:32:19.495304 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 02:32:19.495523 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":954,"mutex_wait_us":53,"thread_start_us":106,"threads_started":1}
I20260619 02:32:19.495682 32033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:19.489315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:44865 (local address 127.0.0.1:33860)
0619 02:32:19.490146 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:19.490204 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:19.490341 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:19.491057 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:19.491063 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:19.491085 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:32:19.491273 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:19.491281 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.492849 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.492852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:19.494104 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:19.494111 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.494983 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.494987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:19.495106 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:19.495199 (+ 93us) client_negotiation.cc:769] Sending connection context
0619 02:32:19.495313 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 02:32:19.495526 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":768,"mutex_wait_us":154,"spinlock_wait_cycles":2432,"thread_start_us":106,"threads_started":1}
I20260619 02:32:19.496186 32036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:19.489242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:46419 (local address 127.0.0.1:47662)
0619 02:32:19.490148 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:19.490202 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:19.490341 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:19.491378 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:19.491382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:19.491399 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:32:19.491485 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:19.491494 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.492543 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.492549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:19.493966 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:19.493975 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.495113 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.495117 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:19.495300 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:19.495327 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 02:32:19.495382 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 02:32:19.496073 (+ 691us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":110,"spinlock_wait_cycles":10368,"thread_start_us":102,"threads_started":1}
I20260619 02:32:19.496421 32037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:19.490088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37096 (local address 127.17.96.193:32897)
0619 02:32:19.490609 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:19.490625 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:19.490648 (+ 23us) server_negotiation.cc:407] Connection header received
0619 02:32:19.490779 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:19.490786 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:19.491078 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:19.491260 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:32:19.491537 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.492653 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.494223 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.494990 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.495931 (+ 941us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:19.496083 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:19.496126 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 02:32:19.496200 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":65,"threads_started":1}
I20260619 02:32:19.499110 31822 tablet_service.cc:1467] Processing CreateTablet for tablet 7bde563597724c6787dae7aff30aa60b (DEFAULT_TABLE table=test-table [id=a719e41f6bc5417a9f3c85068285fbf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:32:19.499181 31956 tablet_service.cc:1467] Processing CreateTablet for tablet 7bde563597724c6787dae7aff30aa60b (DEFAULT_TABLE table=test-table [id=a719e41f6bc5417a9f3c85068285fbf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:32:19.499181 31688 tablet_service.cc:1467] Processing CreateTablet for tablet 7bde563597724c6787dae7aff30aa60b (DEFAULT_TABLE table=test-table [id=a719e41f6bc5417a9f3c85068285fbf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:32:19.500164 31822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bde563597724c6787dae7aff30aa60b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:32:19.500201 31688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bde563597724c6787dae7aff30aa60b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:32:19.500207 31956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bde563597724c6787dae7aff30aa60b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:32:19.506733 32039 tablet_bootstrap.cc:492] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074: Bootstrap starting.
I20260619 02:32:19.508231 32040 tablet_bootstrap.cc:492] T 7bde563597724c6787dae7aff30aa60b P 5ec16fb3eb8a43fd9d0a863f7ce89e7a: Bootstrap starting.
I20260619 02:32:19.508415 32041 tablet_bootstrap.cc:492] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9: Bootstrap starting.
I20260619 02:32:19.509447 32039 tablet_bootstrap.cc:654] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074: Neither blocks nor log segments found. Creating new log.
I20260619 02:32:19.510281 32039 log.cc:824] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074: Log is configured to *not* fsync() on all Append() calls
I20260619 02:32:19.510409 32041 tablet_bootstrap.cc:654] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9: Neither blocks nor log segments found. Creating new log.
I20260619 02:32:19.510878 32040 tablet_bootstrap.cc:654] T 7bde563597724c6787dae7aff30aa60b P 5ec16fb3eb8a43fd9d0a863f7ce89e7a: Neither blocks nor log segments found. Creating new log.
I20260619 02:32:19.511293 32041 log.cc:824] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9: Log is configured to *not* fsync() on all Append() calls
I20260619 02:32:19.511654 32040 log.cc:824] T 7bde563597724c6787dae7aff30aa60b P 5ec16fb3eb8a43fd9d0a863f7ce89e7a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:32:19.512614 32039 tablet_bootstrap.cc:492] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074: No bootstrap required, opened a new log
I20260619 02:32:19.512857 32039 ts_tablet_manager.cc:1397] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260619 02:32:19.513260 32041 tablet_bootstrap.cc:492] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9: No bootstrap required, opened a new log
I20260619 02:32:19.513468 32041 ts_tablet_manager.cc:1397] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260619 02:32:19.513571 32040 tablet_bootstrap.cc:492] T 7bde563597724c6787dae7aff30aa60b P 5ec16fb3eb8a43fd9d0a863f7ce89e7a: No bootstrap required, opened a new log
I20260619 02:32:19.513722 32040 ts_tablet_manager.cc:1397] T 7bde563597724c6787dae7aff30aa60b P 5ec16fb3eb8a43fd9d0a863f7ce89e7a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 02:32:19.516664 32041 raft_consensus.cc:348] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:19.516882 32041 raft_consensus.cc:374] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:32:19.516920 32041 raft_consensus.cc:729] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa0f20ec8cad41f3934b52f1c3ce7bc9, State: Initialized, Role: FOLLOWER
I20260619 02:32:19.516930 32039 raft_consensus.cc:348] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:19.517179 32039 raft_consensus.cc:374] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:32:19.517228 32039 raft_consensus.cc:729] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86f29e57a9774074b91d28ad2d342074, State: Initialized, Role: FOLLOWER
I20260619 02:32:19.517388 32041 consensus_queue.cc:260] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:19.517449 32040 raft_consensus.cc:348] T 7bde563597724c6787dae7aff30aa60b P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:19.517678 32040 raft_consensus.cc:374] T 7bde563597724c6787dae7aff30aa60b P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:32:19.517728 32040 raft_consensus.cc:729] T 7bde563597724c6787dae7aff30aa60b P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec16fb3eb8a43fd9d0a863f7ce89e7a, State: Initialized, Role: FOLLOWER
I20260619 02:32:19.517655 32039 consensus_queue.cc:260] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:19.518229 31754 heartbeater.cc:503] Master 127.17.96.254:38995 was elected leader, sending a full tablet report...
I20260619 02:32:19.518191 32040 consensus_queue.cc:260] T 7bde563597724c6787dae7aff30aa60b P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:19.518497 32041 ts_tablet_manager.cc:1428] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260619 02:32:19.518591 32039 ts_tablet_manager.cc:1428] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260619 02:32:19.518923 31888 heartbeater.cc:503] Master 127.17.96.254:38995 was elected leader, sending a full tablet report...
I20260619 02:32:19.519659 32040 ts_tablet_manager.cc:1428] T 7bde563597724c6787dae7aff30aa60b P 5ec16fb3eb8a43fd9d0a863f7ce89e7a: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260619 02:32:19.520011 32022 heartbeater.cc:503] Master 127.17.96.254:38995 was elected leader, sending a full tablet report...
I20260619 02:32:19.633394 32046 raft_consensus.cc:482] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:32:19.633688 32046 raft_consensus.cc:504] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:19.634963 32046 leader_election.cc:288] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ec16fb3eb8a43fd9d0a863f7ce89e7a (127.17.96.195:46419), aa0f20ec8cad41f3934b52f1c3ce7bc9 (127.17.96.193:32897)
I20260619 02:32:19.640213 32035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:19.635368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:49809 (local address 127.17.96.195:46419)
0619 02:32:19.635497 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:19.635501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:19.635901 (+ 400us) server_negotiation.cc:407] Connection header received
0619 02:32:19.635987 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:19.635989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:19.636036 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:19.636096 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:32:19.636370 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.637081 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.638807 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.639432 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.639567 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:19.639628 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:19.639971 (+ 343us) server_negotiation.cc:299] Negotiation successful
0619 02:32:19.640048 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 02:32:19.640218 31890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:19.635279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:32897 (local address 127.17.96.194:57485)
0619 02:32:19.635415 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:19.635434 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:19.635538 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:19.635870 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:19.635873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:19.635901 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:32:19.636057 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:19.636063 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.637157 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.637163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:19.638674 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:19.638685 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.639593 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.639598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:19.639793 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:19.639838 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 02:32:19.639929 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 02:32:19.640041 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":204,"spinlock_wait_cycles":8960}
I20260619 02:32:19.640225 32048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:19.635286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:46419 (local address 127.17.96.194:49809)
0619 02:32:19.635767 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:19.635778 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:19.635899 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:19.636137 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:19.636143 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:19.636166 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:32:19.636261 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:19.636266 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.637184 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.637187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:19.638674 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:19.638685 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.639593 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.639598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:19.639788 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:19.639838 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 02:32:19.639929 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 02:32:19.640041 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":286,"spinlock_wait_cycles":384,"thread_start_us":130,"threads_started":1}
I20260619 02:32:19.640378 32037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:19.635385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:57485 (local address 127.17.96.193:32897)
0619 02:32:19.635518 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:19.635525 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:19.635546 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:32:19.635710 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:19.635712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:19.635759 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:19.635830 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:32:19.636204 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.637027 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.638808 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.639433 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.639567 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:19.639622 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:19.640172 (+ 550us) server_negotiation.cc:299] Negotiation successful
0619 02:32:19.640262 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:32:19.641036 31708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bde563597724c6787dae7aff30aa60b" candidate_uuid: "86f29e57a9774074b91d28ad2d342074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" is_pre_election: true
I20260619 02:32:19.641189 31976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bde563597724c6787dae7aff30aa60b" candidate_uuid: "86f29e57a9774074b91d28ad2d342074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" is_pre_election: true
I20260619 02:32:19.641393 31708 raft_consensus.cc:2454] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86f29e57a9774074b91d28ad2d342074 in term 0.
I20260619 02:32:19.641480 31976 raft_consensus.cc:2454] T 7bde563597724c6787dae7aff30aa60b P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86f29e57a9774074b91d28ad2d342074 in term 0.
I20260619 02:32:19.641969 31775 leader_election.cc:302] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5ec16fb3eb8a43fd9d0a863f7ce89e7a, 86f29e57a9774074b91d28ad2d342074, aa0f20ec8cad41f3934b52f1c3ce7bc9; no voters:
I20260619 02:32:19.642289 32046 raft_consensus.cc:2781] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:32:19.642381 32046 raft_consensus.cc:482] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:32:19.642423 32046 raft_consensus.cc:3037] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:32:19.643489 32046 raft_consensus.cc:504] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:19.643867 32046 leader_election.cc:288] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074 [CANDIDATE]: Term 1 election: Requested vote from peers 5ec16fb3eb8a43fd9d0a863f7ce89e7a (127.17.96.195:46419), aa0f20ec8cad41f3934b52f1c3ce7bc9 (127.17.96.193:32897)
I20260619 02:32:19.644208 31976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bde563597724c6787dae7aff30aa60b" candidate_uuid: "86f29e57a9774074b91d28ad2d342074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a"
I20260619 02:32:19.644271 31708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bde563597724c6787dae7aff30aa60b" candidate_uuid: "86f29e57a9774074b91d28ad2d342074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9"
I20260619 02:32:19.644345 31976 raft_consensus.cc:3037] T 7bde563597724c6787dae7aff30aa60b P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:32:19.644421 31708 raft_consensus.cc:3037] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:32:19.645517 31708 raft_consensus.cc:2454] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86f29e57a9774074b91d28ad2d342074 in term 1.
I20260619 02:32:19.645538 31976 raft_consensus.cc:2454] T 7bde563597724c6787dae7aff30aa60b P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86f29e57a9774074b91d28ad2d342074 in term 1.
I20260619 02:32:19.645861 31777 leader_election.cc:302] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86f29e57a9774074b91d28ad2d342074, aa0f20ec8cad41f3934b52f1c3ce7bc9; no voters:
I20260619 02:32:19.646113 32046 raft_consensus.cc:2781] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:32:19.646338 32046 raft_consensus.cc:686] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074 [term 1 LEADER]: Becoming Leader. State: Replica: 86f29e57a9774074b91d28ad2d342074, State: Running, Role: LEADER
I20260619 02:32:19.646687 32046 consensus_queue.cc:237] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:19.650117 31563 catalog_manager.cc:5495] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86f29e57a9774074b91d28ad2d342074 (127.17.96.194). New cstate: current_term: 1 leader_uuid: "86f29e57a9774074b91d28ad2d342074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } health_report { overall_health: UNKNOWN } } }
I20260619 02:32:19.687146 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:19.683190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33876 (local address 127.17.96.194:44865)
0619 02:32:19.683374 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:19.683380 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:19.683401 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:32:19.683521 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:19.683524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:19.683585 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:19.683718 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:32:19.684040 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.684598 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.685497 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:19.685725 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:19.685893 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:19.686227 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:32:19.686690 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:32:19.686838 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:19.686904 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 02:32:19.686979 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260619 02:32:19.687320 32023 tablet.cc:2369] T 7bde563597724c6787dae7aff30aa60b P 5ec16fb3eb8a43fd9d0a863f7ce89e7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:32:19.693358 31708 raft_consensus.cc:1261] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Refusing update from remote peer 86f29e57a9774074b91d28ad2d342074: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:32:19.693358 31976 raft_consensus.cc:1261] T 7bde563597724c6787dae7aff30aa60b P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [term 1 FOLLOWER]: Refusing update from remote peer 86f29e57a9774074b91d28ad2d342074: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:32:19.694245 32046 consensus_queue.cc:1035] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:32:19.694485 32049 consensus_queue.cc:1035] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:32:19.704177 32052 mvcc.cc:205] Tried to move back new op lower bound from 7298401647376154624 to 7298401647195406336. Current Snapshot: MvccSnapshot[applied={T|T < 7298401647376154624 or (T in {7298401647376154624})}]
I20260619 02:32:19.708829 32053 mvcc.cc:205] Tried to move back new op lower bound from 7298401647376154624 to 7298401647195406336. Current Snapshot: MvccSnapshot[applied={T|T < 7298401647376154624}]
I20260619 02:32:19.715824 31561 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:32:19.716094 31561 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:32:19.719205 31561 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50: Sending DeleteTablet for 3 replicas of tablet 7bde563597724c6787dae7aff30aa60b
I20260619 02:32:19.720319 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 31625
I20260619 02:32:19.720321 31956 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bde563597724c6787dae7aff30aa60b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:32:19 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:47662
I20260619 02:32:19.720372 31688 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bde563597724c6787dae7aff30aa60b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:32:19 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:37096
I20260619 02:32:19.722565 32061 tablet_replica.cc:330] T 7bde563597724c6787dae7aff30aa60b P 5ec16fb3eb8a43fd9d0a863f7ce89e7a: stopping tablet replica
I20260619 02:32:19.722530 31822 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bde563597724c6787dae7aff30aa60b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:32:19 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:33860
I20260619 02:32:19.723053 32061 raft_consensus.cc:2229] T 7bde563597724c6787dae7aff30aa60b P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:32:19.723418 32061 raft_consensus.cc:2258] T 7bde563597724c6787dae7aff30aa60b P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:32:19.723979 32062 tablet_replica.cc:330] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074: stopping tablet replica
I20260619 02:32:19.724327 32062 raft_consensus.cc:2229] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:32:19.724743 32062 raft_consensus.cc:2258] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:32:19.725471 32061 ts_tablet_manager.cc:1905] T 7bde563597724c6787dae7aff30aa60b P 5ec16fb3eb8a43fd9d0a863f7ce89e7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:32:19.725976 32062 ts_tablet_manager.cc:1905] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:32:19.728766 32062 ts_tablet_manager.cc:1918] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:32:19.728794 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 31759
W20260619 02:32:19.728844 31549 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.96.193:32897 (error 108)
I20260619 02:32:19.728917 32062 log.cc:1196] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/7bde563597724c6787dae7aff30aa60b
I20260619 02:32:19.730103 32061 ts_tablet_manager.cc:1918] T 7bde563597724c6787dae7aff30aa60b P 5ec16fb3eb8a43fd9d0a863f7ce89e7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:32:19.730307 32061 log.cc:1196] T 7bde563597724c6787dae7aff30aa60b P 5ec16fb3eb8a43fd9d0a863f7ce89e7a: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/7bde563597724c6787dae7aff30aa60b
I20260619 02:32:19.730741 32061 ts_tablet_manager.cc:1939] T 7bde563597724c6787dae7aff30aa60b P 5ec16fb3eb8a43fd9d0a863f7ce89e7a: Deleting consensus metadata
I20260619 02:32:19.732014 31547 catalog_manager.cc:4841] TS 5ec16fb3eb8a43fd9d0a863f7ce89e7a (127.17.96.195:46419): tablet 7bde563597724c6787dae7aff30aa60b (table test-table [id=a719e41f6bc5417a9f3c85068285fbf4]) successfully deleted
I20260619 02:32:19.734813 32036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:19.734378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:32897 (local address 127.0.0.1:37104)
0619 02:32:19.734601 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:19.734688 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.193:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260619 02:32:19.734838 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 31893
W20260619 02:32:19.735014 31549 catalog_manager.cc:4584] TS aa0f20ec8cad41f3934b52f1c3ce7bc9 (127.17.96.193:32897): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7bde563597724c6787dae7aff30aa60b: Network error: Client connection negotiation failed: client connection to 127.17.96.193:32897: connect: Connection refused (error 111)
I20260619 02:32:19.736160 32036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:19.735874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:44865 (local address 127.0.0.1:33892)
0619 02:32:19.736001 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:19.736066 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.194:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260619 02:32:19.740808 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 31531
I20260619 02:32:19.747634 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:38995
--webserver_interface=127.17.96.254
--webserver_port=44783
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:36409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:38995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:32:19.863952 32064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:19.864262 32064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:19.864306 32064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:19.868778 32064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:32:19.868866 32064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:19.868887 32064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:19.868906 32064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:32:19.868920 32064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:32:19.874607 32064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:36409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:38995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:38995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=44783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:19.875815 32064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:19.876892 32064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:19.890502 32073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:19.890489 32071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:19.890570 32064 server_base.cc:1034] running on GCE node
W20260619 02:32:19.890489 32070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:19.891486 32064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:19.892777 32064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:19.893945 32064 hybrid_clock.cc:648] HybridClock initialized: now 1781836339893916 us; error 53 us; skew 500 ppm
Jun 19 02:32:19 dist-test-slave-4qw6 krb5kdc[31518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836339, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:19.896816 32064 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:32:19.897890 32064 webserver.cc:460] Webserver started at http://127.17.96.254:44783/ using document root <none> and password file <none>
I20260619 02:32:19.898531 32064 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:19.898634 32064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:19.902864 32064 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260619 02:32:19.905282 32080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:19.906687 32064 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:32:19.906847 32064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "42db01878520488a833834b8996abb50"
format_stamp: "Formatted at 2026-06-19 02:32:18 on dist-test-slave-4qw6"
server_key: "a816f0b3c385997e86c877325a0f0f24"
server_key_iv: "f3c0352120e52bdf37a2dfd81a9e584d"
server_key_version: "encryptionkey@0"
I20260619 02:32:19.907492 32064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:19.927316 32064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:19.930881 32064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:19.931118 32064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:19.938817 32064 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:38995
I20260619 02:32:19.939147 32132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:38995 every 8 connection(s)
I20260619 02:32:19.940032 32064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:32:19.944732 32133 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:32:19.944976 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 32064
I20260619 02:32:19.945451 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:32897
--local_ip_for_outbound_sockets=127.17.96.193
--tserver_master_addrs=127.17.96.254:38995
--webserver_port=38311
--webserver_interface=127.17.96.193
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:36409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:32:19.946727 32133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50: Bootstrap starting.
I20260619 02:32:19.964196 32133 log.cc:824] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50: Log is configured to *not* fsync() on all Append() calls
I20260619 02:32:19.973218 32133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:32:19.973670 32133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50: Bootstrap complete.
I20260619 02:32:19.977929 32133 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42db01878520488a833834b8996abb50" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38995 } }
I20260619 02:32:19.978397 32133 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42db01878520488a833834b8996abb50, State: Initialized, Role: FOLLOWER
I20260619 02:32:19.978964 32133 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42db01878520488a833834b8996abb50" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38995 } }
I20260619 02:32:19.979112 32133 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:32:19.979206 32133 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:32:19.979326 32133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:32:19.981379 32133 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42db01878520488a833834b8996abb50" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38995 } }
I20260619 02:32:19.981786 32133 leader_election.cc:302] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42db01878520488a833834b8996abb50; no voters:
I20260619 02:32:19.982172 32133 leader_election.cc:288] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 02:32:19.982264 32137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 02:32:19.982483 32137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 2 LEADER]: Becoming Leader. State: Replica: 42db01878520488a833834b8996abb50, State: Running, Role: LEADER
I20260619 02:32:19.982843 32137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42db01878520488a833834b8996abb50" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38995 } }
I20260619 02:32:19.983484 32133 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:32:19.983836 32137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42db01878520488a833834b8996abb50. Latest consensus state: current_term: 2 leader_uuid: "42db01878520488a833834b8996abb50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42db01878520488a833834b8996abb50" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38995 } } }
I20260619 02:32:19.983960 32137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [sys.catalog]: This master's current role is: LEADER
I20260619 02:32:19.984328 32141 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:32:19.984362 32138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "42db01878520488a833834b8996abb50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42db01878520488a833834b8996abb50" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38995 } } }
I20260619 02:32:19.984591 32138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [sys.catalog]: This master's current role is: LEADER
I20260619 02:32:19.989396 32141 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:32:19.990182 32141 catalog_manager.cc:1259] Loaded cluster ID: e42653a319c34db7b69e4bbf17ba83c1
I20260619 02:32:19.990269 32141 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:32:19.991417 32141 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260619 02:32:19.994290 32141 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260619 02:32:19.999014 32141 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:32:19.999852 32141 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50: Loaded TSK: 0
I20260619 02:32:20.000916 32141 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:32:20.076666 32135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:20.077037 32135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:20.077128 32135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:20.081562 32135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:20.081688 32135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:20.081842 32135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:32:20.088353 32135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:36409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:32897
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=38311
--tserver_master_addrs=127.17.96.254:38995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:20.089500 32135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:20.090622 32135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:20.100899 32161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:20.100914 32160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:20.101171 32163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:20.101332 32135 server_base.cc:1034] running on GCE node
I20260619 02:32:20.101878 32135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:20.102595 32135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:20.103801 32135 hybrid_clock.cc:648] HybridClock initialized: now 1781836340103768 us; error 53 us; skew 500 ppm
Jun 19 02:32:20 dist-test-slave-4qw6 krb5kdc[31518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836340, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:20.106900 32135 init.cc:377] Logged in from keytab as kudu/127.17.96.193@KRBTEST.COM (short username kudu)
I20260619 02:32:20.107939 32135 webserver.cc:460] Webserver started at http://127.17.96.193:38311/ using document root <none> and password file <none>
I20260619 02:32:20.108554 32135 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:20.108675 32135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:20.113111 32135 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.002s
I20260619 02:32:20.115598 32170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:20.116977 32135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:32:20.117137 32135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9"
format_stamp: "Formatted at 2026-06-19 02:32:18 on dist-test-slave-4qw6"
server_key: "a7fbbc45af4bd27a50fe135dcd2573ce"
server_key_iv: "09474d6bddf9ff8824b6803bcf149d2d"
server_key_version: "encryptionkey@0"
I20260619 02:32:20.117739 32135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:20.147132 32135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:20.150846 32135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:20.151086 32135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:20.151790 32135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:32:20.153014 32177 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 02:32:20.157665 32135 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 02:32:20.157797 32135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260619 02:32:20.157907 32135 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 02:32:20.159704 32135 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 02:32:20.159785 32135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260619 02:32:20.160059 32177 tablet_bootstrap.cc:492] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9: Bootstrap starting.
I20260619 02:32:20.170964 32135 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:32897
I20260619 02:32:20.171036 32284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:32897 every 8 connection(s)
I20260619 02:32:20.172008 32135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:32:20.173148 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 32135
I20260619 02:32:20.173591 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:44865
--local_ip_for_outbound_sockets=127.17.96.194
--tserver_master_addrs=127.17.96.254:38995
--webserver_port=37521
--webserver_interface=127.17.96.194
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:36409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:32:20.180801 32177 log.cc:824] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9: Log is configured to *not* fsync() on all Append() calls
Jun 19 02:32:20 dist-test-slave-4qw6 krb5kdc[31518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836340, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:20.191372 32177 tablet_bootstrap.cc:492] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:32:20.191861 32287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:20.174487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:57731 (local address 127.17.96.254:38995)
0619 02:32:20.175020 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:20.175045 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:20.175809 (+ 764us) server_negotiation.cc:407] Connection header received
0619 02:32:20.177352 (+ 1543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:20.177376 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:20.177882 (+ 506us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:20.178251 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:20.179390 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.180468 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.182058 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.182595 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.182766 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:20.185863 (+ 3097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:20.185869 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:20.185882 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:20.185914 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:20.188306 (+ 2392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:20.188903 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:20.188910 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:20.188915 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:20.189056 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:20.189423 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:20.189428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:20.189429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:20.189948 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:20.190108 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:20.190660 (+ 552us) server_negotiation.cc:299] Negotiation successful
0619 02:32:20.190929 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":127,"threads_started":1}
I20260619 02:32:20.191879 32288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:20.174514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.193:57731)
0619 02:32:20.175567 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:20.175635 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:20.176806 (+ 1171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:20.178319 (+ 1513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:20.178326 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:20.178699 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:20.179178 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:20.179192 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.180688 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.180691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:20.181907 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:20.181921 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.182988 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.182992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:20.183179 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:20.183902 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:20.183948 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:20.185593 (+ 1645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:20.188449 (+ 2856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:20.188456 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:20.188469 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:20.188768 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:20.189166 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:20.189170 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:20.189172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:20.189319 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:20.190071 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:20.190078 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:20.190332 (+ 254us) client_negotiation.cc:769] Sending connection context
0619 02:32:20.190531 (+ 199us) client_negotiation.cc:240] Negotiation successful
0619 02:32:20.190911 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":806,"thread_start_us":168,"threads_started":1}
I20260619 02:32:20.191941 32177 tablet_bootstrap.cc:492] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9: Bootstrap complete.
I20260619 02:32:20.193420 32177 ts_tablet_manager.cc:1397] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9: Time spent bootstrapping tablet: real 0.034s user 0.030s sys 0.000s
I20260619 02:32:20.194693 32285 heartbeater.cc:347] Connected to a master server at 127.17.96.254:38995
I20260619 02:32:20.194967 32285 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:20.195500 32285 heartbeater.cc:511] Master 127.17.96.254:38995 requested a full tablet report, sending...
I20260619 02:32:20.196870 32177 raft_consensus.cc:348] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:20.197448 32177 raft_consensus.cc:729] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa0f20ec8cad41f3934b52f1c3ce7bc9, State: Initialized, Role: FOLLOWER
I20260619 02:32:20.198035 32177 consensus_queue.cc:260] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:20.198280 32097 ts_manager.cc:194] Registered new tserver with Master: aa0f20ec8cad41f3934b52f1c3ce7bc9 (127.17.96.193:32897)
I20260619 02:32:20.199234 32177 ts_tablet_manager.cc:1428] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260619 02:32:20.202293 32097 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:57731
I20260619 02:32:20.205127 32285 heartbeater.cc:503] Master 127.17.96.254:38995 was elected leader, sending a full tablet report...
Jun 19 02:32:20 dist-test-slave-4qw6 krb5kdc[31518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836339, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for kudu/127.17.96.193@KRBTEST.COM
I20260619 02:32:20.214058 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:20.201606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:32897 (local address 127.0.0.1:37108)
0619 02:32:20.202595 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:20.202656 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:20.202835 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:20.203473 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:20.203479 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:20.203945 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:20.204112 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:20.204121 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.205419 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.205423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:20.206515 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:20.206528 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.207331 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.207335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:20.207456 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:20.208057 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:20.208072 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:20.209684 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:20.211951 (+ 2267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:20.211962 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:20.211967 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:20.212301 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:20.212678 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:20.212682 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:20.212684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:20.212777 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:20.213289 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:20.213295 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:20.213437 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 02:32:20.213599 (+ 162us) client_negotiation.cc:240] Negotiation successful
0619 02:32:20.213863 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"thread_start_us":164,"threads_started":1}
I20260619 02:32:20.214058 32293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:20.201776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37108 (local address 127.17.96.193:32897)
0619 02:32:20.202288 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:20.202304 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:20.202790 (+ 486us) server_negotiation.cc:407] Connection header received
0619 02:32:20.203011 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:20.203018 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:20.203293 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:20.203448 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:20.204266 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.205304 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.206667 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.207200 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.207304 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:20.209890 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:20.209909 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:20.209920 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:20.209943 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:20.211829 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:20.212440 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:20.212445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:20.212446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:20.212573 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:20.212875 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:20.212879 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:20.212880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:20.213169 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:20.213320 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:20.213705 (+ 385us) server_negotiation.cc:299] Negotiation successful
0619 02:32:20.213910 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":132,"threads_started":1}
I20260619 02:32:20.215097 32219 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bde563597724c6787dae7aff30aa60b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:32:19 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:37108
I20260619 02:32:20.216073 32294 tablet_replica.cc:330] stopping tablet replica
I20260619 02:32:20.216437 32294 raft_consensus.cc:2229] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:32:20.216734 32294 raft_consensus.cc:2258] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:32:20.217912 32294 ts_tablet_manager.cc:1905] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:32:20.222966 32294 ts_tablet_manager.cc:1918] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:32:20.223100 32294 log.cc:1196] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/wal/wals/7bde563597724c6787dae7aff30aa60b
I20260619 02:32:20.223373 32294 ts_tablet_manager.cc:1939] T 7bde563597724c6787dae7aff30aa60b P aa0f20ec8cad41f3934b52f1c3ce7bc9: Deleting consensus metadata
I20260619 02:32:20.225556 32083 catalog_manager.cc:4841] TS aa0f20ec8cad41f3934b52f1c3ce7bc9 (127.17.96.193:32897): tablet 7bde563597724c6787dae7aff30aa60b (table test-table [id=a719e41f6bc5417a9f3c85068285fbf4]) successfully deleted
W20260619 02:32:20.304028 32289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:20.304337 32289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:20.304414 32289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:20.308782 32289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:20.308862 32289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:20.308974 32289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:32:20.314922 32289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:36409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:44865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=37521
--tserver_master_addrs=127.17.96.254:38995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:20.316141 32289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:20.317286 32289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:20.327651 32300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:20.327673 32299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:20.327670 32302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:20.327729 32289 server_base.cc:1034] running on GCE node
I20260619 02:32:20.328370 32289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:20.329156 32289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:20.330329 32289 hybrid_clock.cc:648] HybridClock initialized: now 1781836340330314 us; error 30 us; skew 500 ppm
Jun 19 02:32:20 dist-test-slave-4qw6 krb5kdc[31518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836340, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:20.333240 32289 init.cc:377] Logged in from keytab as kudu/127.17.96.194@KRBTEST.COM (short username kudu)
I20260619 02:32:20.334250 32289 webserver.cc:460] Webserver started at http://127.17.96.194:37521/ using document root <none> and password file <none>
I20260619 02:32:20.334805 32289 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:20.334882 32289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:20.338884 32289 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:32:20.341065 32309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:20.342474 32289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:32:20.342667 32289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "86f29e57a9774074b91d28ad2d342074"
format_stamp: "Formatted at 2026-06-19 02:32:19 on dist-test-slave-4qw6"
server_key: "a86468564d28b96e3c410f9ab79419a6"
server_key_iv: "9dc979305a04cf0518e8a427cf18309f"
server_key_version: "encryptionkey@0"
I20260619 02:32:20.343233 32289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:20.359814 32289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:20.363245 32289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:20.363428 32289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:20.364065 32289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:32:20.365231 32316 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260619 02:32:20.369549 32316 ts_tablet_manager.cc:1852] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260619 02:32:20.370002 32316 ts_tablet_manager.cc:1905] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:32:20.374111 32316 ts_tablet_manager.cc:1918] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:32:20.374330 32316 ts_tablet_manager.cc:1939] T 7bde563597724c6787dae7aff30aa60b P 86f29e57a9774074b91d28ad2d342074: Deleting consensus metadata
I20260619 02:32:20.374678 32289 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260619 02:32:20.374760 32289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.010s user 0.001s sys 0.000s
I20260619 02:32:20.374881 32289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:32:20.374924 32289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:20.384395 32289 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:44865
I20260619 02:32:20.384447 32423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:44865 every 8 connection(s)
I20260619 02:32:20.385473 32289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:32:20.393608 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 32289
I20260619 02:32:20.394073 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:46419
--local_ip_for_outbound_sockets=127.17.96.195
--tserver_master_addrs=127.17.96.254:38995
--webserver_port=36531
--webserver_interface=127.17.96.195
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:36409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:32:20 dist-test-slave-4qw6 krb5kdc[31518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836340, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:20.400759 32287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:20.387485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:35543 (local address 127.17.96.254:38995)
0619 02:32:20.387668 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:20.387672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:20.388635 (+ 963us) server_negotiation.cc:407] Connection header received
0619 02:32:20.389581 (+ 946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:20.389584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:20.389635 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:20.389715 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:20.391020 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.391562 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.392691 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.392925 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.393020 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:20.396328 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:20.396332 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:20.396334 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:20.396364 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:20.398148 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:20.398776 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:20.398783 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:20.398788 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:20.398899 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:20.399293 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:20.399298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:20.399302 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:20.399611 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:20.399736 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:20.400297 (+ 561us) server_negotiation.cc:299] Negotiation successful
0619 02:32:20.400489 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 02:32:20.401453 32426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:20.387944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.194:35543)
0619 02:32:20.388467 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:20.388512 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:20.389374 (+ 862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:20.389965 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:20.389973 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:20.390346 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:20.390857 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:20.390871 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.391681 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.391683 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:20.392573 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:20.392582 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.393112 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.393116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:20.393284 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:20.394063 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:20.394081 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:20.396077 (+ 1996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:20.398319 (+ 2242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:20.398325 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:20.398340 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:20.398646 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:20.399015 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:20.399017 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:20.399019 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:20.399187 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:20.399731 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:20.399736 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:20.400018 (+ 282us) client_negotiation.cc:769] Sending connection context
0619 02:32:20.400204 (+ 186us) client_negotiation.cc:240] Negotiation successful
0619 02:32:20.400496 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":141,"threads_started":1}
I20260619 02:32:20.403031 32424 heartbeater.cc:347] Connected to a master server at 127.17.96.254:38995
I20260619 02:32:20.403316 32424 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:20.403849 32424 heartbeater.cc:511] Master 127.17.96.254:38995 requested a full tablet report, sending...
I20260619 02:32:20.405066 32097 ts_manager.cc:194] Registered new tserver with Master: 86f29e57a9774074b91d28ad2d342074 (127.17.96.194:44865)
I20260619 02:32:20.405684 32097 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:35543
W20260619 02:32:20.513020 32427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:20.513329 32427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:20.513407 32427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:20.517705 32427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:20.517869 32427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:20.517963 32427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:32:20.523942 32427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:36409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:46419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=36531
--tserver_master_addrs=127.17.96.254:38995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:20.525108 32427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:20.526203 32427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:32:20.536489 32427 server_base.cc:1034] running on GCE node
W20260619 02:32:20.536442 32432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:20.536435 32433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:20.536446 32435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:20.537104 32427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:20.537796 32427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:20.539008 32427 hybrid_clock.cc:648] HybridClock initialized: now 1781836340538988 us; error 35 us; skew 500 ppm
Jun 19 02:32:20 dist-test-slave-4qw6 krb5kdc[31518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836340, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:20.542044 32427 init.cc:377] Logged in from keytab as kudu/127.17.96.195@KRBTEST.COM (short username kudu)
I20260619 02:32:20.543107 32427 webserver.cc:460] Webserver started at http://127.17.96.195:36531/ using document root <none> and password file <none>
I20260619 02:32:20.543726 32427 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:20.543836 32427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:20.548326 32427 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:32:20.550585 32442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:20.552273 32427 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:32:20.552412 32427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a"
format_stamp: "Formatted at 2026-06-19 02:32:19 on dist-test-slave-4qw6"
server_key: "99d101fb6908af95dd44dec776a1e657"
server_key_iv: "6facbd1753831c3e57d6145b7b0212ad"
server_key_version: "encryptionkey@0"
I20260619 02:32:20.553023 32427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:20.588905 32427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:20.592480 32427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:20.592713 32427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:20.593356 32427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:32:20.594395 32427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:32:20.594470 32427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:20.594535 32427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:32:20.594576 32427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:20.606796 32427 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:46419
I20260619 02:32:20.606879 32555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:46419 every 8 connection(s)
I20260619 02:32:20.607888 32427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:32:20.612866 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 32427
Jun 19 02:32:20 dist-test-slave-4qw6 krb5kdc[31518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836340, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:20.622968 32560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:20.613613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55416 (local address 127.17.96.254:38995)
0619 02:32:20.614219 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:20.614223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:20.614234 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:32:20.614266 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:20.614269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:20.614303 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:20.614374 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:20.615305 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.616065 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.617124 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.617495 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.617615 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:20.619155 (+ 1540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:20.619158 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:20.619160 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:20.619184 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:20.621023 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:20.621490 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:20.621494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:20.621497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:20.621616 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:20.621986 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:20.621992 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:20.621995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:20.622204 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:20.622494 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:20.622658 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 02:32:20.622785 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":539,"thread_start_us":114,"threads_started":1}
I20260619 02:32:20.623962 32287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:20.609858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:40397 (local address 127.17.96.254:38995)
0619 02:32:20.610014 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:20.610017 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:20.610865 (+ 848us) server_negotiation.cc:407] Connection header received
0619 02:32:20.612470 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:20.612473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:20.612539 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:20.612649 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:20.614210 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.615050 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.616663 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.617012 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.617127 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:20.620083 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:20.620087 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:20.620089 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:20.620149 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:20.621568 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:20.622249 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:20.622256 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:20.622258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:20.622332 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:20.622760 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:20.622763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:20.622765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:20.623005 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:20.623099 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:20.623705 (+ 606us) server_negotiation.cc:299] Negotiation successful
0619 02:32:20.623822 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":51}
I20260619 02:32:20.624962 32558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:20.610229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.195:40397)
0619 02:32:20.610705 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:20.610749 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:20.612249 (+ 1500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:20.612843 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:20.612851 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:20.613312 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:20.614052 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:20.614069 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.615209 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.615212 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:20.616517 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:20.616527 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.617191 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.617195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:20.617391 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:20.618136 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:20.618153 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:20.619903 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:20.621695 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:20.621702 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:20.621721 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:20.622133 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:20.622429 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:20.622433 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:20.622435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:20.622658 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:20.623099 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:20.623108 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:20.623426 (+ 318us) client_negotiation.cc:769] Sending connection context
0619 02:32:20.623653 (+ 227us) client_negotiation.cc:240] Negotiation successful
0619 02:32:20.624020 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":128,"threads_started":1}
I20260619 02:32:20.626149 32556 heartbeater.cc:347] Connected to a master server at 127.17.96.254:38995
I20260619 02:32:20.626441 32556 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:20.626974 32556 heartbeater.cc:511] Master 127.17.96.254:38995 requested a full tablet report, sending...
I20260619 02:32:20.628058 32097 ts_manager.cc:194] Registered new tserver with Master: 5ec16fb3eb8a43fd9d0a863f7ce89e7a (127.17.96.195:46419)
I20260619 02:32:20.628710 32097 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:40397
I20260619 02:32:20.636893 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:32:20.641616 32287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:20.637588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55418 (local address 127.17.96.254:38995)
0619 02:32:20.637810 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:20.637814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:20.638005 (+ 191us) server_negotiation.cc:407] Connection header received
0619 02:32:20.638233 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:20.638236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:20.638346 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:20.638439 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:32:20.638686 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.639195 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.640061 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.640354 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.640434 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:20.640730 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:32:20.641161 (+ 431us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:32:20.641253 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:20.641376 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 02:32:20.641428 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 02:32:20.642442 32097 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:55418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:32:20.645840 32097 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:32:20.654181 32219 tablet_service.cc:1467] Processing CreateTablet for tablet d3930cde7921428eb2b86119e39e1f78 (DEFAULT_TABLE table=test-table [id=1a162f970ade48608da178a968f45c92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:32:20.654899 32219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3930cde7921428eb2b86119e39e1f78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:32:20.657918 32177 tablet_bootstrap.cc:492] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9: Bootstrap starting.
I20260619 02:32:20.658900 32177 tablet_bootstrap.cc:654] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9: Neither blocks nor log segments found. Creating new log.
I20260619 02:32:20.659107 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:20.653044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:46419 (local address 127.0.0.1:47678)
0619 02:32:20.653228 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:20.653244 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:20.653345 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:20.654586 (+ 1241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:20.654589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:20.654607 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:32:20.654745 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:20.654752 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.656100 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.656103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:20.657400 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:20.657411 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.658633 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.658637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:20.658738 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:20.658794 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 02:32:20.658871 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 02:32:20.658965 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260619 02:32:20.659365 32565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:20.653201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47678 (local address 127.17.96.195:46419)
0619 02:32:20.653888 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:20.653906 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:20.653931 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:32:20.654020 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:20.654027 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:20.654285 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:20.654477 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:32:20.654885 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.655930 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.657597 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.658504 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.658619 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:20.658776 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:20.659062 (+ 286us) server_negotiation.cc:299] Negotiation successful
0619 02:32:20.659199 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":134,"threads_started":1}
I20260619 02:32:20.659979 32566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:20.653479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:44865 (local address 127.0.0.1:33902)
0619 02:32:20.653831 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:20.653845 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:20.653931 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:20.654717 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:20.654720 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:20.654734 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:32:20.654819 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:20.654827 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.656471 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.656475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:20.657605 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:20.657613 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.659640 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.659644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:20.659736 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:20.659770 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 02:32:20.659826 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 02:32:20.659883 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":96,"threads_started":1}
I20260619 02:32:20.660549 32567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:20.653943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33902 (local address 127.17.96.194:44865)
0619 02:32:20.654289 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:20.654300 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:20.654323 (+ 23us) server_negotiation.cc:407] Connection header received
0619 02:32:20.654397 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:20.654402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:20.654572 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:20.654730 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:32:20.654916 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.656330 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.657900 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.659488 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.660195 (+ 707us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:20.660310 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:20.660355 (+ 45us) server_negotiation.cc:299] Negotiation successful
0619 02:32:20.660426 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":128,"threads_started":1}
I20260619 02:32:20.662285 32177 tablet_bootstrap.cc:492] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9: No bootstrap required, opened a new log
I20260619 02:32:20.661799 32358 tablet_service.cc:1467] Processing CreateTablet for tablet d3930cde7921428eb2b86119e39e1f78 (DEFAULT_TABLE table=test-table [id=1a162f970ade48608da178a968f45c92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:32:20.662392 32177 ts_tablet_manager.cc:1397] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260619 02:32:20.662431 32490 tablet_service.cc:1467] Processing CreateTablet for tablet d3930cde7921428eb2b86119e39e1f78 (DEFAULT_TABLE table=test-table [id=1a162f970ade48608da178a968f45c92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:32:20.662940 32358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3930cde7921428eb2b86119e39e1f78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:32:20.662879 32177 raft_consensus.cc:348] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:20.663000 32177 raft_consensus.cc:374] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:32:20.663036 32177 raft_consensus.cc:729] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa0f20ec8cad41f3934b52f1c3ce7bc9, State: Initialized, Role: FOLLOWER
I20260619 02:32:20.663178 32177 consensus_queue.cc:260] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:20.663235 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3930cde7921428eb2b86119e39e1f78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:32:20.663430 32177 ts_tablet_manager.cc:1428] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 02:32:20.667332 32316 tablet_bootstrap.cc:492] T d3930cde7921428eb2b86119e39e1f78 P 86f29e57a9774074b91d28ad2d342074: Bootstrap starting.
I20260619 02:32:20.670118 32316 tablet_bootstrap.cc:654] T d3930cde7921428eb2b86119e39e1f78 P 86f29e57a9774074b91d28ad2d342074: Neither blocks nor log segments found. Creating new log.
I20260619 02:32:20.671052 32316 log.cc:824] T d3930cde7921428eb2b86119e39e1f78 P 86f29e57a9774074b91d28ad2d342074: Log is configured to *not* fsync() on all Append() calls
I20260619 02:32:20.672094 32570 tablet_bootstrap.cc:492] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a: Bootstrap starting.
I20260619 02:32:20.673101 32316 tablet_bootstrap.cc:492] T d3930cde7921428eb2b86119e39e1f78 P 86f29e57a9774074b91d28ad2d342074: No bootstrap required, opened a new log
W20260619 02:32:20.673249 32286 tablet.cc:2369] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:32:20.673309 32316 ts_tablet_manager.cc:1397] T d3930cde7921428eb2b86119e39e1f78 P 86f29e57a9774074b91d28ad2d342074: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 02:32:20.674404 32570 tablet_bootstrap.cc:654] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a: Neither blocks nor log segments found. Creating new log.
I20260619 02:32:20.675496 32570 log.cc:824] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:32:20.676633 32316 raft_consensus.cc:348] T d3930cde7921428eb2b86119e39e1f78 P 86f29e57a9774074b91d28ad2d342074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:20.676880 32316 raft_consensus.cc:374] T d3930cde7921428eb2b86119e39e1f78 P 86f29e57a9774074b91d28ad2d342074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:32:20.676965 32316 raft_consensus.cc:729] T d3930cde7921428eb2b86119e39e1f78 P 86f29e57a9774074b91d28ad2d342074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86f29e57a9774074b91d28ad2d342074, State: Initialized, Role: FOLLOWER
I20260619 02:32:20.677392 32316 consensus_queue.cc:260] T d3930cde7921428eb2b86119e39e1f78 P 86f29e57a9774074b91d28ad2d342074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:20.677582 32570 tablet_bootstrap.cc:492] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a: No bootstrap required, opened a new log
I20260619 02:32:20.677826 32570 ts_tablet_manager.cc:1397] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 02:32:20.678118 32424 heartbeater.cc:503] Master 127.17.96.254:38995 was elected leader, sending a full tablet report...
I20260619 02:32:20.678359 32316 ts_tablet_manager.cc:1428] T d3930cde7921428eb2b86119e39e1f78 P 86f29e57a9774074b91d28ad2d342074: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 02:32:20.681865 32570 raft_consensus.cc:348] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:20.682183 32570 raft_consensus.cc:374] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:32:20.682243 32570 raft_consensus.cc:729] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec16fb3eb8a43fd9d0a863f7ce89e7a, State: Initialized, Role: FOLLOWER
I20260619 02:32:20.682669 32570 consensus_queue.cc:260] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:20.683380 32556 heartbeater.cc:503] Master 127.17.96.254:38995 was elected leader, sending a full tablet report...
I20260619 02:32:20.683774 32570 ts_tablet_manager.cc:1428] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260619 02:32:20.748051 32573 raft_consensus.cc:482] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:32:20.748317 32573 raft_consensus.cc:504] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:20.749547 32573 leader_election.cc:288] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86f29e57a9774074b91d28ad2d342074 (127.17.96.194:44865), aa0f20ec8cad41f3934b52f1c3ce7bc9 (127.17.96.193:32897)
I20260619 02:32:20.755062 32574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:20.749982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:50433 (local address 127.17.96.193:32897)
0619 02:32:20.750298 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:20.750301 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:20.750314 (+ 13us) server_negotiation.cc:407] Connection header received
0619 02:32:20.750363 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:20.750365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:20.750453 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:20.750572 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:32:20.750959 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.751751 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.753641 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.754394 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.754509 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:20.754673 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:20.754835 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 02:32:20.754913 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":131,"threads_started":1}
I20260619 02:32:20.755061 32567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:20.749959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:49721 (local address 127.17.96.194:44865)
0619 02:32:20.750089 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:20.750094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:20.750461 (+ 367us) server_negotiation.cc:407] Connection header received
0619 02:32:20.750525 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:20.750531 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:20.750587 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:20.750657 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:32:20.750950 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.751759 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.753641 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.754308 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.754492 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:20.754551 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:20.754835 (+ 284us) server_negotiation.cc:299] Negotiation successful
0619 02:32:20.754913 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:32:20.755136 32558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:20.749881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:32897 (local address 127.17.96.195:50433)
0619 02:32:20.749997 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:20.750015 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:20.750135 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:20.750602 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:20.750604 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:20.750618 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:32:20.750798 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:20.750807 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.751878 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.751881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:20.753451 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:20.753463 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.754491 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.754494 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:20.754663 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:20.754713 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 02:32:20.754796 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 02:32:20.754955 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":251}
I20260619 02:32:20.755143 32575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:20.749886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:44865 (local address 127.17.96.195:49721)
0619 02:32:20.750345 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:20.750356 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:20.750446 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:20.750682 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:20.750685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:20.750697 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:32:20.750826 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:20.750834 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.751878 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.751881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:20.753449 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:20.753463 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.754481 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.754485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:20.754598 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:20.754713 (+ 115us) client_negotiation.cc:769] Sending connection context
0619 02:32:20.754796 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 02:32:20.754955 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":349,"thread_start_us":100,"threads_started":1}
I20260619 02:32:20.755868 32238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3930cde7921428eb2b86119e39e1f78" candidate_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" is_pre_election: true
I20260619 02:32:20.756011 32378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3930cde7921428eb2b86119e39e1f78" candidate_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86f29e57a9774074b91d28ad2d342074" is_pre_election: true
I20260619 02:32:20.756264 32238 raft_consensus.cc:2454] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ec16fb3eb8a43fd9d0a863f7ce89e7a in term 0.
I20260619 02:32:20.756389 32378 raft_consensus.cc:2454] T d3930cde7921428eb2b86119e39e1f78 P 86f29e57a9774074b91d28ad2d342074 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ec16fb3eb8a43fd9d0a863f7ce89e7a in term 0.
I20260619 02:32:20.756898 32445 leader_election.cc:302] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5ec16fb3eb8a43fd9d0a863f7ce89e7a, 86f29e57a9774074b91d28ad2d342074, aa0f20ec8cad41f3934b52f1c3ce7bc9; no voters:
I20260619 02:32:20.757225 32573 raft_consensus.cc:2781] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:32:20.757337 32573 raft_consensus.cc:482] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:32:20.757396 32573 raft_consensus.cc:3037] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:32:20.758607 32573 raft_consensus.cc:504] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:20.759003 32573 leader_election.cc:288] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [CANDIDATE]: Term 1 election: Requested vote from peers 86f29e57a9774074b91d28ad2d342074 (127.17.96.194:44865), aa0f20ec8cad41f3934b52f1c3ce7bc9 (127.17.96.193:32897)
I20260619 02:32:20.759431 32238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3930cde7921428eb2b86119e39e1f78" candidate_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9"
I20260619 02:32:20.759588 32238 raft_consensus.cc:3037] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:32:20.759596 32378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3930cde7921428eb2b86119e39e1f78" candidate_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86f29e57a9774074b91d28ad2d342074"
I20260619 02:32:20.759732 32378 raft_consensus.cc:3037] T d3930cde7921428eb2b86119e39e1f78 P 86f29e57a9774074b91d28ad2d342074 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:32:20.760846 32238 raft_consensus.cc:2454] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec16fb3eb8a43fd9d0a863f7ce89e7a in term 1.
I20260619 02:32:20.761032 32378 raft_consensus.cc:2454] T d3930cde7921428eb2b86119e39e1f78 P 86f29e57a9774074b91d28ad2d342074 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec16fb3eb8a43fd9d0a863f7ce89e7a in term 1.
I20260619 02:32:20.761206 32445 leader_election.cc:302] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ec16fb3eb8a43fd9d0a863f7ce89e7a, aa0f20ec8cad41f3934b52f1c3ce7bc9; no voters:
I20260619 02:32:20.761413 32573 raft_consensus.cc:2781] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:32:20.761683 32573 raft_consensus.cc:686] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [term 1 LEADER]: Becoming Leader. State: Replica: 5ec16fb3eb8a43fd9d0a863f7ce89e7a, State: Running, Role: LEADER
I20260619 02:32:20.762063 32573 consensus_queue.cc:237] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:20.765390 32097 catalog_manager.cc:5495] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ec16fb3eb8a43fd9d0a863f7ce89e7a (127.17.96.195). New cstate: current_term: 1 leader_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } health_report { overall_health: UNKNOWN } } }
I20260619 02:32:20.793437 32565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:20.789791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47682 (local address 127.17.96.195:46419)
0619 02:32:20.789933 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:20.789938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:20.789953 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:32:20.790057 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:20.790059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:20.790112 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:20.790200 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:32:20.790402 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.790931 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.791831 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:20.792072 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:20.792204 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:20.792525 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:32:20.792983 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:32:20.793128 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:20.793190 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 02:32:20.793254 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:32:20.799863 32378 raft_consensus.cc:1261] T d3930cde7921428eb2b86119e39e1f78 P 86f29e57a9774074b91d28ad2d342074 [term 1 FOLLOWER]: Refusing update from remote peer 5ec16fb3eb8a43fd9d0a863f7ce89e7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:32:20.799862 32238 raft_consensus.cc:1261] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Refusing update from remote peer 5ec16fb3eb8a43fd9d0a863f7ce89e7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:32:20.800650 32576 consensus_queue.cc:1035] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:32:20.800879 32573 consensus_queue.cc:1035] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:32:20.807686 32580 mvcc.cc:205] Tried to move back new op lower bound from 7298401651907641344 to 7298401651764105216. Current Snapshot: MvccSnapshot[applied={T|T < 7298401651907641344}]
I20260619 02:32:20.817732 32096 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:55418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:32:20.817983 32096 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:55418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:32:20.820683 32096 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50: Sending DeleteTablet for 3 replicas of tablet d3930cde7921428eb2b86119e39e1f78
I20260619 02:32:20.821564 32490 tablet_service.cc:1514] Processing DeleteTablet for tablet d3930cde7921428eb2b86119e39e1f78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:32:20 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:47678
I20260619 02:32:20.821599 32358 tablet_service.cc:1514] Processing DeleteTablet for tablet d3930cde7921428eb2b86119e39e1f78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:32:20 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:33902
I20260619 02:32:20.821848 32219 tablet_service.cc:1514] Processing DeleteTablet for tablet d3930cde7921428eb2b86119e39e1f78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:32:20 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:37108
I20260619 02:32:20.822088 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 32135
I20260619 02:32:20.822316 32590 tablet_replica.cc:330] T d3930cde7921428eb2b86119e39e1f78 P 86f29e57a9774074b91d28ad2d342074: stopping tablet replica
I20260619 02:32:20.822739 32590 raft_consensus.cc:2229] T d3930cde7921428eb2b86119e39e1f78 P 86f29e57a9774074b91d28ad2d342074 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:32:20.823112 32590 raft_consensus.cc:2258] T d3930cde7921428eb2b86119e39e1f78 P 86f29e57a9774074b91d28ad2d342074 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:32:20.822166 32589 tablet_replica.cc:330] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a: stopping tablet replica
I20260619 02:32:20.824664 32589 raft_consensus.cc:2229] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:32:20.824895 32590 ts_tablet_manager.cc:1905] T d3930cde7921428eb2b86119e39e1f78 P 86f29e57a9774074b91d28ad2d342074: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:32:20.825109 32589 raft_consensus.cc:2258] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:32:20.826802 32589 ts_tablet_manager.cc:1905] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:32:20.828039 32590 ts_tablet_manager.cc:1918] T d3930cde7921428eb2b86119e39e1f78 P 86f29e57a9774074b91d28ad2d342074: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:32:20.828162 32590 log.cc:1196] T d3930cde7921428eb2b86119e39e1f78 P 86f29e57a9774074b91d28ad2d342074: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/d3930cde7921428eb2b86119e39e1f78
I20260619 02:32:20.828526 32590 ts_tablet_manager.cc:1939] T d3930cde7921428eb2b86119e39e1f78 P 86f29e57a9774074b91d28ad2d342074: Deleting consensus metadata
I20260619 02:32:20.829635 32589 ts_tablet_manager.cc:1918] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:32:20.829741 32589 log.cc:1196] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/d3930cde7921428eb2b86119e39e1f78
I20260619 02:32:20.830056 32084 catalog_manager.cc:4841] TS 86f29e57a9774074b91d28ad2d342074 (127.17.96.194:44865): tablet d3930cde7921428eb2b86119e39e1f78 (table test-table [id=1a162f970ade48608da178a968f45c92]) successfully deleted
I20260619 02:32:20.830147 32589 ts_tablet_manager.cc:1939] T d3930cde7921428eb2b86119e39e1f78 P 5ec16fb3eb8a43fd9d0a863f7ce89e7a: Deleting consensus metadata
W20260619 02:32:20.830510 32083 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.96.193:32897 (error 108)
I20260619 02:32:20.830910 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 32289
I20260619 02:32:20.831918 32081 catalog_manager.cc:4841] TS 5ec16fb3eb8a43fd9d0a863f7ce89e7a (127.17.96.195:46419): tablet d3930cde7921428eb2b86119e39e1f78 (table test-table [id=1a162f970ade48608da178a968f45c92]) successfully deleted
I20260619 02:32:20.833709 32566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:20.833281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:32897 (local address 127.0.0.1:37112)
0619 02:32:20.833500 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:20.833584 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.193:32897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260619 02:32:20.833931 32083 catalog_manager.cc:4584] TS aa0f20ec8cad41f3934b52f1c3ce7bc9 (127.17.96.193:32897): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d3930cde7921428eb2b86119e39e1f78: Network error: Client connection negotiation failed: client connection to 127.17.96.193:32897: connect: Connection refused (error 111)
I20260619 02:32:20.837571 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 32427
I20260619 02:32:20.843353 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 32064
I20260619 02:32:20.849915 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:38995
--webserver_interface=127.17.96.254
--webserver_port=44783
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:36409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:38995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:32:20.967064 32593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:20.967399 32593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:20.967497 32593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:20.971710 32593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:32:20.971832 32593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:20.971868 32593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:20.971910 32593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:32:20.971946 32593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:32:20.977815 32593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:36409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:38995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:38995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=44783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:20.979040 32593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:20.980113 32593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:20.989149 32598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:20.989100 32601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:20.989298 32593 server_base.cc:1034] running on GCE node
W20260619 02:32:20.989122 32599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:20.990192 32593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:20.991353 32593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:20.992568 32593 hybrid_clock.cc:648] HybridClock initialized: now 1781836340992556 us; error 31 us; skew 500 ppm
Jun 19 02:32:20 dist-test-slave-4qw6 krb5kdc[31518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836340, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:20.995558 32593 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:32:20.996575 32593 webserver.cc:460] Webserver started at http://127.17.96.254:44783/ using document root <none> and password file <none>
I20260619 02:32:20.997215 32593 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:20.997309 32593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:21.001408 32593 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:32:21.003767 32608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:21.005040 32593 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260619 02:32:21.005205 32593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "42db01878520488a833834b8996abb50"
format_stamp: "Formatted at 2026-06-19 02:32:18 on dist-test-slave-4qw6"
server_key: "a816f0b3c385997e86c877325a0f0f24"
server_key_iv: "f3c0352120e52bdf37a2dfd81a9e584d"
server_key_version: "encryptionkey@0"
I20260619 02:32:21.005856 32593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:21.025028 32593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:21.028780 32593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:21.029006 32593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:21.037097 32593 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:38995
I20260619 02:32:21.037415 32660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:38995 every 8 connection(s)
I20260619 02:32:21.038470 32593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:32:21.043118 32661 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:32:21.044929 32661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50: Bootstrap starting.
I20260619 02:32:21.047837 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 32593
I20260619 02:32:21.048321 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:32897
--local_ip_for_outbound_sockets=127.17.96.193
--tserver_master_addrs=127.17.96.254:38995
--webserver_port=38311
--webserver_interface=127.17.96.193
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:36409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:32:21.062750 32661 log.cc:824] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50: Log is configured to *not* fsync() on all Append() calls
I20260619 02:32:21.074517 32661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:32:21.074949 32661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50: Bootstrap complete.
I20260619 02:32:21.079121 32661 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42db01878520488a833834b8996abb50" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38995 } }
I20260619 02:32:21.079559 32661 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42db01878520488a833834b8996abb50, State: Initialized, Role: FOLLOWER
I20260619 02:32:21.080123 32661 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42db01878520488a833834b8996abb50" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38995 } }
I20260619 02:32:21.080281 32661 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:32:21.080370 32661 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:32:21.080487 32661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 2 FOLLOWER]: Advancing to term 3
I20260619 02:32:21.082545 32661 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42db01878520488a833834b8996abb50" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38995 } }
I20260619 02:32:21.082918 32661 leader_election.cc:302] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42db01878520488a833834b8996abb50; no voters:
I20260619 02:32:21.083271 32661 leader_election.cc:288] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 02:32:21.083434 32665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 02:32:21.083699 32665 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [term 3 LEADER]: Becoming Leader. State: Replica: 42db01878520488a833834b8996abb50, State: Running, Role: LEADER
I20260619 02:32:21.084025 32665 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42db01878520488a833834b8996abb50" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38995 } }
I20260619 02:32:21.084568 32661 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:32:21.085475 32666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "42db01878520488a833834b8996abb50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42db01878520488a833834b8996abb50" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38995 } } }
I20260619 02:32:21.085618 32666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [sys.catalog]: This master's current role is: LEADER
I20260619 02:32:21.085919 32667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42db01878520488a833834b8996abb50. Latest consensus state: current_term: 3 leader_uuid: "42db01878520488a833834b8996abb50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42db01878520488a833834b8996abb50" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 38995 } } }
I20260619 02:32:21.086000 32667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42db01878520488a833834b8996abb50 [sys.catalog]: This master's current role is: LEADER
I20260619 02:32:21.086529 32674 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:32:21.091859 32674 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:32:21.092612 32674 catalog_manager.cc:1259] Loaded cluster ID: e42653a319c34db7b69e4bbf17ba83c1
I20260619 02:32:21.092681 32674 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260619 02:32:21.093801 32674 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f98785d59ad google::LogMessage::Fail() at ??:0
@ 0x7f98785d888a google::LogMessage::SendToLog() at ??:0
@ 0x7f98785d54de google::LogMessage::Flush() at ??:0
@ 0x7f98785d6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9882f14fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f9882f15d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9882f12579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f9882f56640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f987a8273d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f987a827c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f987a8293f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f987a818512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f987be5c6db start_thread at ??:0
@ 0x7f98770d361f clone at ??:0
W20260619 02:32:21.194535 32663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:21.194828 32663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:21.194875 32663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:21.199137 32663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:21.199213 32663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:21.199304 32663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:32:21.205016 32663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:36409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:32897
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=38311
--tserver_master_addrs=127.17.96.254:38995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:21.206347 32663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:21.207466 32663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:21.218086 32702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:21.218228 32705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:21.218113 32703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:21.218394 32663 server_base.cc:1034] running on GCE node
I20260619 02:32:21.219061 32663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:21.220022 32663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:21.221261 32663 hybrid_clock.cc:648] HybridClock initialized: now 1781836341221232 us; error 40 us; skew 500 ppm
Jun 19 02:32:21 dist-test-slave-4qw6 krb5kdc[31518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836341, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:21.225168 32663 init.cc:377] Logged in from keytab as kudu/127.17.96.193@KRBTEST.COM (short username kudu)
I20260619 02:32:21.226615 32663 webserver.cc:460] Webserver started at http://127.17.96.193:38311/ using document root <none> and password file <none>
I20260619 02:32:21.227408 32663 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:21.227528 32663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:21.233130 32663 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:32:21.236006 32712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:21.237522 32663 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:32:21.237682 32663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9"
format_stamp: "Formatted at 2026-06-19 02:32:18 on dist-test-slave-4qw6"
server_key: "a7fbbc45af4bd27a50fe135dcd2573ce"
server_key_iv: "09474d6bddf9ff8824b6803bcf149d2d"
server_key_version: "encryptionkey@0"
I20260619 02:32:21.238344 32663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:21.262089 32663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:21.265833 32663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:21.266062 32663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:21.266772 32663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:32:21.268002 32719 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 02:32:21.273970 32663 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 02:32:21.274088 32663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260619 02:32:21.274204 32663 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 02:32:21.276247 32663 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 02:32:21.276342 32663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260619 02:32:21.276662 32719 tablet_bootstrap.cc:492] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9: Bootstrap starting.
I20260619 02:32:21.288810 32663 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:32897
I20260619 02:32:21.289105 358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:32897 every 8 connection(s)
I20260619 02:32:21.290200 32663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:32:21.296311 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 32663
I20260619 02:32:21.296815 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:44865
--local_ip_for_outbound_sockets=127.17.96.194
--tserver_master_addrs=127.17.96.254:38995
--webserver_port=37521
--webserver_interface=127.17.96.194
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:36409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:32:21.301194 32719 log.cc:824] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9: Log is configured to *not* fsync() on all Append() calls
I20260619 02:32:21.311797 32719 tablet_bootstrap.cc:492] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:32:21.312330 32719 tablet_bootstrap.cc:492] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9: Bootstrap complete.
I20260619 02:32:21.313887 32719 ts_tablet_manager.cc:1397] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9: Time spent bootstrapping tablet: real 0.037s user 0.029s sys 0.008s
I20260619 02:32:21.317584 32719 raft_consensus.cc:348] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:21.318298 32719 raft_consensus.cc:729] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa0f20ec8cad41f3934b52f1c3ce7bc9, State: Initialized, Role: FOLLOWER
I20260619 02:32:21.318934 32719 consensus_queue.cc:260] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:21.320266 32719 ts_tablet_manager.cc:1428] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260619 02:32:21.345964 361 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:32:21.292483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.193:49865)
0619 02:32:21.293049 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:21.293110 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:21.294673 (+ 1563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:21.344920 (+ 50247us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":180,"threads_started":1}
W20260619 02:32:21.346545 359 heartbeater.cc:644] Failed to heartbeat to 127.17.96.254:38995 (0 consecutive failures): Network error: Failed to ping master at 127.17.96.254:38995: Client connection negotiation failed: client connection to 127.17.96.254:38995: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 02:32:21.347401 361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:21.347071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.193:59951)
0619 02:32:21.347232 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:21.347319 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 02:32:21.348165 361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:21.347901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.193:57009)
0619 02:32:21.348026 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:21.348081 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 02:32:21.348409 359 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:32:21.426731 362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:21.427042 362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:21.427090 362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:21.431533 362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:21.431612 362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:21.431731 362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:32:21.437603 362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:36409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:44865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=37521
--tserver_master_addrs=127.17.96.254:38995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:21.438812 362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:21.439903 362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:21.450331 370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:21.450289 372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:21.450316 369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:21.450628 362 server_base.cc:1034] running on GCE node
I20260619 02:32:21.451104 362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:21.451772 362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:21.452950 362 hybrid_clock.cc:648] HybridClock initialized: now 1781836341452935 us; error 30 us; skew 500 ppm
Jun 19 02:32:21 dist-test-slave-4qw6 krb5kdc[31518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836341, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:21.455878 362 init.cc:377] Logged in from keytab as kudu/127.17.96.194@KRBTEST.COM (short username kudu)
I20260619 02:32:21.456861 362 webserver.cc:460] Webserver started at http://127.17.96.194:37521/ using document root <none> and password file <none>
I20260619 02:32:21.457481 362 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:21.457551 362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:21.461925 362 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260619 02:32:21.464079 379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:21.465548 362 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260619 02:32:21.465704 362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "86f29e57a9774074b91d28ad2d342074"
format_stamp: "Formatted at 2026-06-19 02:32:19 on dist-test-slave-4qw6"
server_key: "a86468564d28b96e3c410f9ab79419a6"
server_key_iv: "9dc979305a04cf0518e8a427cf18309f"
server_key_version: "encryptionkey@0"
I20260619 02:32:21.466414 362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:21.482939 362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:21.486179 362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:21.486402 362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:21.487056 362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:32:21.487983 362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:32:21.488061 362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:21.488152 362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:32:21.488197 362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:21.497843 362 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:44865
I20260619 02:32:21.497886 492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:44865 every 8 connection(s)
I20260619 02:32:21.498963 362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:32:21.501772 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 362
I20260619 02:32:21.502274 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:46419
--local_ip_for_outbound_sockets=127.17.96.195
--tserver_master_addrs=127.17.96.254:38995
--webserver_port=36531
--webserver_interface=127.17.96.195
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:36409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:32:21.503397 495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:21.501306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.194:48399)
0619 02:32:21.501948 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:21.502093 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":160,"threads_started":1}
W20260619 02:32:21.504036 493 heartbeater.cc:644] Failed to heartbeat to 127.17.96.254:38995 (0 consecutive failures): Network error: Failed to ping master at 127.17.96.254:38995: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
I20260619 02:32:21.504750 495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:21.504502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.194:54925)
0619 02:32:21.504624 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:21.504680 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260619 02:32:21.505508 495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:21.505237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.194:54965)
0619 02:32:21.505371 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:21.505431 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 02:32:21.505698 493 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:32:21.618178 496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:21.618470 496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:21.618516 496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:21.622789 496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:21.622874 496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:21.622938 496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:32:21.628760 496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:36409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:46419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=36531
--tserver_master_addrs=127.17.96.254:38995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:21.629980 496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:21.631071 496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:21.641683 501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:21.641669 496 server_base.cc:1034] running on GCE node
W20260619 02:32:21.641687 504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:21.641642 502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:21.642421 496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:21.643188 496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:21.644402 496 hybrid_clock.cc:648] HybridClock initialized: now 1781836341644364 us; error 54 us; skew 500 ppm
Jun 19 02:32:21 dist-test-slave-4qw6 krb5kdc[31518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836341, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:21.647660 496 init.cc:377] Logged in from keytab as kudu/127.17.96.195@KRBTEST.COM (short username kudu)
I20260619 02:32:21.648720 496 webserver.cc:460] Webserver started at http://127.17.96.195:36531/ using document root <none> and password file <none>
I20260619 02:32:21.649298 496 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:21.649374 496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:21.653622 496 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:32:21.655979 511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:21.657399 496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:32:21.657657 496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a"
format_stamp: "Formatted at 2026-06-19 02:32:19 on dist-test-slave-4qw6"
server_key: "99d101fb6908af95dd44dec776a1e657"
server_key_iv: "6facbd1753831c3e57d6145b7b0212ad"
server_key_version: "encryptionkey@0"
I20260619 02:32:21.658321 496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:21.701589 496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:21.705116 496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:21.705390 496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:21.706138 496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:32:21.707089 496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:32:21.707137 496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:21.707214 496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:32:21.707247 496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:21.717677 496 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:46419
I20260619 02:32:21.717705 624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:46419 every 8 connection(s)
I20260619 02:32:21.718947 496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:32:21.720930 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 496
I20260619 02:32:21.723703 628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:21.721627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.195:58631)
0619 02:32:21.722316 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:21.722448 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":185,"threads_started":1}
W20260619 02:32:21.724318 625 heartbeater.cc:644] Failed to heartbeat to 127.17.96.254:38995 (0 consecutive failures): Network error: Failed to ping master at 127.17.96.254:38995: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
I20260619 02:32:21.725060 628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:21.724820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.195:60355)
0619 02:32:21.724935 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:21.724987 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 02:32:21.725812 628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:21.725563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.195:39559)
0619 02:32:21.725663 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:21.725706 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260619 02:32:21.726056 625 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 02:32:22.349638 629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:22.349117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.193:33231)
0619 02:32:22.349445 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:22.349525 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":80,"threads_started":1}
I20260619 02:32:22.506857 630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:22.506396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.194:51113)
0619 02:32:22.506678 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:22.506742 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":89,"threads_started":1}
I20260619 02:32:22.548707 631 raft_consensus.cc:482] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:32:22.548959 631 raft_consensus.cc:504] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:22.550246 631 leader_election.cc:288] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5ec16fb3eb8a43fd9d0a863f7ce89e7a (127.17.96.195:46419), 86f29e57a9774074b91d28ad2d342074 (127.17.96.194:44865)
Jun 19 02:32:22 dist-test-slave-4qw6 krb5kdc[31518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836341, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.194@KRBTEST.COM
Jun 19 02:32:22 dist-test-slave-4qw6 krb5kdc[31518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836341, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.195@KRBTEST.COM
I20260619 02:32:22.565292 629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:22.550449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:46419 (local address 127.17.96.193:56443)
0619 02:32:22.551021 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:22.551039 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:22.551157 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:22.552773 (+ 1616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:22.552778 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:22.553377 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:22.553902 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:22.553918 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:22.555226 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:22.555232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:22.556147 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:22.556157 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:22.556884 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:22.556888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:22.557115 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:22.557927 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:22.557945 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:22.559939 (+ 1994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:22.562346 (+ 2407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:22.562348 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:22.562350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:22.562931 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:22.563322 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:22.563326 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:22.563328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:22.563467 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:22.564269 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:22.564277 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:22.564507 (+ 230us) client_negotiation.cc:769] Sending connection context
0619 02:32:22.564696 (+ 189us) client_negotiation.cc:240] Negotiation successful
0619 02:32:22.565053 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":133,"spinlock_wait_cycles":42624}
I20260619 02:32:22.565304 634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:22.551010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:42713 (local address 127.17.96.194:44865)
0619 02:32:22.551313 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:22.551330 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:22.551356 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:32:22.551973 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:22.551978 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:22.552391 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:22.552706 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:22.554095 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:22.555067 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:22.556284 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:22.556752 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:22.556936 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:22.559877 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:22.559884 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:22.559896 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:22.559927 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:22.562175 (+ 2248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:22.563371 (+ 1196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:22.563380 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:22.563388 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:22.563550 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:22.563825 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:22.563828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:22.563830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:22.564249 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:22.564392 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:22.564813 (+ 421us) server_negotiation.cc:299] Negotiation successful
0619 02:32:22.565107 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":95,"threads_started":1}
I20260619 02:32:22.565311 633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:22.550645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:56443 (local address 127.17.96.195:46419)
0619 02:32:22.551048 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:22.551065 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:22.551124 (+ 59us) server_negotiation.cc:407] Connection header received
0619 02:32:22.551925 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:22.551931 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:22.552394 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:22.552706 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:22.554088 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:22.555072 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:22.556281 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:22.556751 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:22.556939 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:22.560142 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:22.560151 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:22.560167 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:22.560204 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:22.562175 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:22.563065 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:22.563072 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:22.563077 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:22.563216 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:22.563575 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:22.563579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:22.563580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:22.564146 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:22.564300 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:22.564813 (+ 513us) server_negotiation.cc:299] Negotiation successful
0619 02:32:22.565074 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":133,"threads_started":1}
I20260619 02:32:22.565654 632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:22.550551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:44865 (local address 127.17.96.193:42713)
0619 02:32:22.551100 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:22.551112 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:22.551177 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:22.552979 (+ 1802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:22.552982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:22.553375 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:22.553901 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:22.553918 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:22.555242 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:22.555245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:22.556150 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:22.556159 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:22.556884 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:22.556888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:22.557111 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:22.557862 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:22.557879 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:22.559619 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:22.562319 (+ 2700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:22.562324 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:22.562335 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:22.562931 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:22.563642 (+ 711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:22.563644 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:22.563646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:22.563721 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:22.564362 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:22.564365 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:22.564507 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 02:32:22.564696 (+ 189us) client_negotiation.cc:240] Negotiation successful
0619 02:32:22.565180 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":267,"spinlock_wait_cycles":23040,"thread_start_us":81,"threads_started":1}
W20260619 02:32:22.567901 32716 leader_election.cc:341] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 86f29e57a9774074b91d28ad2d342074 (127.17.96.194:44865): Not found: Tablet not found: d3930cde7921428eb2b86119e39e1f78
W20260619 02:32:22.568115 32713 leader_election.cc:341] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ec16fb3eb8a43fd9d0a863f7ce89e7a (127.17.96.195:46419): Not found: Tablet not found: d3930cde7921428eb2b86119e39e1f78
I20260619 02:32:22.568225 32713 leader_election.cc:302] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa0f20ec8cad41f3934b52f1c3ce7bc9; no voters: 5ec16fb3eb8a43fd9d0a863f7ce89e7a, 86f29e57a9774074b91d28ad2d342074
I20260619 02:32:22.568543 631 raft_consensus.cc:2726] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:32:22.727281 635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:22.726796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.195:47679)
0619 02:32:22.727094 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:22.727173 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":88,"threads_started":1}
I20260619 02:32:23.350989 636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:23.350544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.193:39029)
0619 02:32:23.350830 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:23.350888 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":101,"threads_started":1}
I20260619 02:32:23.508183 637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:23.507726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.194:53847)
0619 02:32:23.508006 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:23.508064 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":87,"threads_started":1}
I20260619 02:32:23.728587 638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:23.728148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.195:36043)
0619 02:32:23.728415 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:23.728474 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":95,"threads_started":1}
I20260619 02:32:24.352281 639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:24.351830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.193:33295)
0619 02:32:24.352121 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:24.352183 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":78,"threads_started":1}
I20260619 02:32:24.509459 640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:24.509007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.194:44799)
0619 02:32:24.509296 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:24.509361 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":85,"threads_started":1}
I20260619 02:32:24.729884 641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:24.729427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.195:40349)
0619 02:32:24.729701 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:24.729782 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":86,"threads_started":1}
I20260619 02:32:24.740868 642 raft_consensus.cc:482] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:32:24.741008 642 raft_consensus.cc:504] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:24.741485 642 leader_election.cc:288] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5ec16fb3eb8a43fd9d0a863f7ce89e7a (127.17.96.195:46419), 86f29e57a9774074b91d28ad2d342074 (127.17.96.194:44865)
W20260619 02:32:24.742353 32713 leader_election.cc:341] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ec16fb3eb8a43fd9d0a863f7ce89e7a (127.17.96.195:46419): Not found: Tablet not found: d3930cde7921428eb2b86119e39e1f78
W20260619 02:32:24.742491 32716 leader_election.cc:341] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 86f29e57a9774074b91d28ad2d342074 (127.17.96.194:44865): Not found: Tablet not found: d3930cde7921428eb2b86119e39e1f78
I20260619 02:32:24.742631 32716 leader_election.cc:302] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa0f20ec8cad41f3934b52f1c3ce7bc9; no voters: 5ec16fb3eb8a43fd9d0a863f7ce89e7a, 86f29e57a9774074b91d28ad2d342074
I20260619 02:32:24.742789 642 raft_consensus.cc:2726] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:32:25.353645 643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:25.353178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.193:54101)
0619 02:32:25.353476 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:25.353543 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":77,"threads_started":1}
I20260619 02:32:25.510838 644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:25.510365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.194:52353)
0619 02:32:25.510669 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:25.510737 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":96,"threads_started":1}
I20260619 02:32:25.731208 645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:25.730753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.195:47759)
0619 02:32:25.731037 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:25.731105 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":94,"threads_started":1}
I20260619 02:32:26.355178 646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:26.354691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.193:39765)
0619 02:32:26.355030 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:26.355084 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":109,"threads_started":1}
I20260619 02:32:26.512218 647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:26.511753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.194:47027)
0619 02:32:26.512054 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:26.512118 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":79,"threads_started":1}
I20260619 02:32:26.732654 648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:26.732190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.195:36929)
0619 02:32:26.732488 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:26.732546 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":104,"threads_started":1}
I20260619 02:32:27.356443 649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:27.356020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.193:39367)
0619 02:32:27.356280 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:27.356342 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":83,"threads_started":1}
I20260619 02:32:27.513623 650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:27.513148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.194:60727)
0619 02:32:27.513447 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:27.513519 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":83,"threads_started":1}
I20260619 02:32:27.690752 651 raft_consensus.cc:482] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:32:27.690922 651 raft_consensus.cc:504] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:27.691395 651 leader_election.cc:288] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5ec16fb3eb8a43fd9d0a863f7ce89e7a (127.17.96.195:46419), 86f29e57a9774074b91d28ad2d342074 (127.17.96.194:44865)
W20260619 02:32:27.692513 32716 leader_election.cc:341] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 86f29e57a9774074b91d28ad2d342074 (127.17.96.194:44865): Not found: Tablet not found: d3930cde7921428eb2b86119e39e1f78
W20260619 02:32:27.692620 32713 leader_election.cc:341] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ec16fb3eb8a43fd9d0a863f7ce89e7a (127.17.96.195:46419): Not found: Tablet not found: d3930cde7921428eb2b86119e39e1f78
I20260619 02:32:27.692736 32713 leader_election.cc:302] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa0f20ec8cad41f3934b52f1c3ce7bc9; no voters: 5ec16fb3eb8a43fd9d0a863f7ce89e7a, 86f29e57a9774074b91d28ad2d342074
I20260619 02:32:27.692950 651 raft_consensus.cc:2726] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:32:27.734237 652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:27.733649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.195:41895)
0619 02:32:27.734023 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:27.734106 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":193,"threads_started":1}
I20260619 02:32:28.357935 653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:28.357380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.193:46361)
0619 02:32:28.357683 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:28.357825 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":87,"threads_started":1}
I20260619 02:32:28.515009 654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:28.514516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.194:35073)
0619 02:32:28.514839 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:28.514908 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":83,"threads_started":1}
I20260619 02:32:28.735648 655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:28.735132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.195:49979)
0619 02:32:28.735478 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:28.735548 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":81,"threads_started":1}
I20260619 02:32:29.359369 656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:29.358898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.193:56203)
0619 02:32:29.359218 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:29.359274 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":90,"threads_started":1}
I20260619 02:32:29.368880 657 raft_consensus.cc:482] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:32:29.369024 657 raft_consensus.cc:504] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:29.369567 657 leader_election.cc:288] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5ec16fb3eb8a43fd9d0a863f7ce89e7a (127.17.96.195:46419), 86f29e57a9774074b91d28ad2d342074 (127.17.96.194:44865)
W20260619 02:32:29.370443 32716 leader_election.cc:341] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 86f29e57a9774074b91d28ad2d342074 (127.17.96.194:44865): Not found: Tablet not found: d3930cde7921428eb2b86119e39e1f78
W20260619 02:32:29.370555 32713 leader_election.cc:341] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ec16fb3eb8a43fd9d0a863f7ce89e7a (127.17.96.195:46419): Not found: Tablet not found: d3930cde7921428eb2b86119e39e1f78
I20260619 02:32:29.370645 32713 leader_election.cc:302] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa0f20ec8cad41f3934b52f1c3ce7bc9; no voters: 5ec16fb3eb8a43fd9d0a863f7ce89e7a, 86f29e57a9774074b91d28ad2d342074
I20260619 02:32:29.370817 657 raft_consensus.cc:2726] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:32:29.516322 658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:29.515901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.194:43621)
0619 02:32:29.516163 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:29.516221 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":75,"threads_started":1}
I20260619 02:32:29.736850 659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:29.736446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.195:36701)
0619 02:32:29.736684 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:29.736739 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":73,"threads_started":1}
I20260619 02:32:30.360705 660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:30.360280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.193:53957)
0619 02:32:30.360544 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:30.360608 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":79,"threads_started":1}
I20260619 02:32:30.517586 661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:30.517155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.194:43995)
0619 02:32:30.517415 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:30.517468 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":76,"threads_started":1}
I20260619 02:32:30.738372 662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:30.737872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.195:39587)
0619 02:32:30.738197 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:30.738266 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":80,"threads_started":1}
I20260619 02:32:31.362162 663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:31.361687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.193:34923)
0619 02:32:31.361977 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:31.362042 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":112,"threads_started":1}
I20260619 02:32:31.518929 664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:31.518473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.194:42853)
0619 02:32:31.518757 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:31.518826 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":86,"threads_started":1}
I20260619 02:32:31.739717 665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:31.739270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.195:54825)
0619 02:32:31.739549 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:31.739613 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":78,"threads_started":1}
I20260619 02:32:32.363437 666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:32.363029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.193:56237)
0619 02:32:32.363279 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:32.363339 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":70,"threads_started":1}
I20260619 02:32:32.520238 667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:32.519796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.194:50323)
0619 02:32:32.520072 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:32.520135 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":93,"threads_started":1}
I20260619 02:32:32.635423 668 raft_consensus.cc:482] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:32:32.635593 668 raft_consensus.cc:504] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ec16fb3eb8a43fd9d0a863f7ce89e7a" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 46419 } } peers { permanent_uuid: "86f29e57a9774074b91d28ad2d342074" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44865 } } peers { permanent_uuid: "aa0f20ec8cad41f3934b52f1c3ce7bc9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 32897 } }
I20260619 02:32:32.636101 668 leader_election.cc:288] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5ec16fb3eb8a43fd9d0a863f7ce89e7a (127.17.96.195:46419), 86f29e57a9774074b91d28ad2d342074 (127.17.96.194:44865)
W20260619 02:32:32.636931 32716 leader_election.cc:341] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 86f29e57a9774074b91d28ad2d342074 (127.17.96.194:44865): Not found: Tablet not found: d3930cde7921428eb2b86119e39e1f78
W20260619 02:32:32.637048 32713 leader_election.cc:341] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ec16fb3eb8a43fd9d0a863f7ce89e7a (127.17.96.195:46419): Not found: Tablet not found: d3930cde7921428eb2b86119e39e1f78
I20260619 02:32:32.637177 32713 leader_election.cc:302] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa0f20ec8cad41f3934b52f1c3ce7bc9; no voters: 5ec16fb3eb8a43fd9d0a863f7ce89e7a, 86f29e57a9774074b91d28ad2d342074
I20260619 02:32:32.637393 668 raft_consensus.cc:2726] T d3930cde7921428eb2b86119e39e1f78 P aa0f20ec8cad41f3934b52f1c3ce7bc9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:32:32.741092 669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:32.740631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.195:33445)
0619 02:32:32.740915 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:32.740981 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":95,"threads_started":1}
I20260619 02:32:33.364743 670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:33.364287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.193:58271)
0619 02:32:33.364572 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:33.364642 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":81,"threads_started":1}
I20260619 02:32:33.521728 671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:33.521224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.194:58823)
0619 02:32:33.521564 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:33.521629 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":156,"threads_started":1}
I20260619 02:32:33.742624 672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:33.742157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.195:40123)
0619 02:32:33.742457 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:33.742523 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":80,"threads_started":1}
I20260619 02:32:34.366290 673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:34.365707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.193:42831)
0619 02:32:34.366058 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:34.366125 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":130,"threads_started":1}
I20260619 02:32:34.523110 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:34.522649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.194:46649)
0619 02:32:34.522908 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:34.522981 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":89,"threads_started":1}
I20260619 02:32:34.744234 675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:34.743668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.195:56429)
0619 02:32:34.744030 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:34.744107 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":93,"threads_started":1}
I20260619 02:32:35.367568 676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:35.367139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.193:53207)
0619 02:32:35.367390 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:35.367451 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":80,"threads_started":1}
I20260619 02:32:35.524551 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:35.524092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.194:59237)
0619 02:32:35.524384 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:35.524451 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":84,"threads_started":1}
I20260619 02:32:35.745642 678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:35.745169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.195:37687)
0619 02:32:35.745464 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:35.745533 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":104,"threads_started":1}
I20260619 02:32:36.369144 679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:36.368721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.193:47095)
0619 02:32:36.368985 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:36.369047 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":84,"threads_started":1}
I20260619 02:32:36.526226 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:36.525700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:38995 (local address 127.17.96.194:55127)
0619 02:32:36.526064 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:36.526129 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":133,"threads_started":1}
I20260619 02:32:36.725287 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 32663
I20260619 02:32:36.731438 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 362
I20260619 02:32:36.737310 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 496
2026-06-19T02:32:36Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19407 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:32:36 dist-test-slave-4qw6 krb5kdc[683](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:32:36 dist-test-slave-4qw6 krb5kdc[683](info): set up 2 sockets
Jun 19 02:32:36 dist-test-slave-4qw6 krb5kdc[683](info): commencing operation
krb5kdc: starting...
W20260619 02:32:37.790112 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:32:37 dist-test-slave-4qw6 krb5kdc[683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:32:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:32:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:32:37.914770 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:40869
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:39763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:40869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:32:38.029706 696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:38.030053 696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:38.030099 696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:38.034472 696 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 02:32:38.034545 696 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 02:32:38.034561 696 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 02:32:38.034576 696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:32:38.034597 696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:38.034616 696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:38.034679 696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:32:38.034711 696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:32:38.040337 696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:39763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:40869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:40869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:38.041589 696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:38.042698 696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:38.051662 702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:38.051704 701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:38.051692 704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:38.051869 696 server_base.cc:1034] running on GCE node
I20260619 02:32:38.052537 696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:38.053719 696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:38.054998 696 hybrid_clock.cc:648] HybridClock initialized: now 1781836358054984 us; error 41 us; skew 500 ppm
Jun 19 02:32:38 dist-test-slave-4qw6 krb5kdc[683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836358, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:38.057971 696 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:32:38.058959 696 webserver.cc:460] Webserver started at http://127.17.96.254:44899/ using document root <none> and password file <none>
I20260619 02:32:38.059553 696 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:38.059662 696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:38.059854 696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:32:38.061990 696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "60800600ea9544229691af08477568a2"
format_stamp: "Formatted at 2026-06-19 02:32:38 on dist-test-slave-4qw6"
server_key: "525651ac22f1f7824b16f1c30cf38818"
server_key_iv: "766d3624adb6bf407434b06fab1d4221"
server_key_version: "encryptionkey@0"
I20260619 02:32:38.062534 696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "60800600ea9544229691af08477568a2"
format_stamp: "Formatted at 2026-06-19 02:32:38 on dist-test-slave-4qw6"
server_key: "525651ac22f1f7824b16f1c30cf38818"
server_key_iv: "766d3624adb6bf407434b06fab1d4221"
server_key_version: "encryptionkey@0"
I20260619 02:32:38.066390 696 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 02:32:38.069032 711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:38.070644 696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:32:38.070801 696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "60800600ea9544229691af08477568a2"
format_stamp: "Formatted at 2026-06-19 02:32:38 on dist-test-slave-4qw6"
server_key: "525651ac22f1f7824b16f1c30cf38818"
server_key_iv: "766d3624adb6bf407434b06fab1d4221"
server_key_version: "encryptionkey@0"
I20260619 02:32:38.070948 696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:38.092903 696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:38.096029 696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:38.096251 696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:38.103669 696 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:40869
I20260619 02:32:38.103952 763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:40869 every 8 connection(s)
I20260619 02:32:38.104730 696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:32:38.107646 764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:32:38.112188 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 696
I20260619 02:32:38.112322 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:32:38.112617 17795 external_mini_cluster.cc:1428] Setting key 787c7b8608dbdda8613cdbe926d9a232
I20260619 02:32:38.114002 764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2: Bootstrap starting.
I20260619 02:32:38.116361 764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2: Neither blocks nor log segments found. Creating new log.
I20260619 02:32:38.117161 764 log.cc:824] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2: Log is configured to *not* fsync() on all Append() calls
I20260619 02:32:38.119608 764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2: No bootstrap required, opened a new log
Jun 19 02:32:38 dist-test-slave-4qw6 krb5kdc[683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:38.125630 764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60800600ea9544229691af08477568a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40869 } }
I20260619 02:32:38.126019 764 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:32:38.126109 764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60800600ea9544229691af08477568a2, State: Initialized, Role: FOLLOWER
I20260619 02:32:38.126706 764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60800600ea9544229691af08477568a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40869 } }
I20260619 02:32:38.126917 764 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:32:38.127012 764 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:32:38.127142 764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:32:38.128358 764 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60800600ea9544229691af08477568a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40869 } }
I20260619 02:32:38.128839 764 leader_election.cc:302] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60800600ea9544229691af08477568a2; no voters:
I20260619 02:32:38.129269 764 leader_election.cc:288] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:32:38.129352 769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:32:38.129585 769 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 1 LEADER]: Becoming Leader. State: Replica: 60800600ea9544229691af08477568a2, State: Running, Role: LEADER
I20260619 02:32:38.129926 769 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60800600ea9544229691af08477568a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40869 } }
I20260619 02:32:38.130867 764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:32:38.131625 770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60800600ea9544229691af08477568a2. Latest consensus state: current_term: 1 leader_uuid: "60800600ea9544229691af08477568a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60800600ea9544229691af08477568a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40869 } } }
I20260619 02:32:38.131765 770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:32:38.132006 767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:38.114224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60184 (local address 127.17.96.254:40869)
0619 02:32:38.114748 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:38.114769 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:38.114809 (+ 40us) server_negotiation.cc:407] Connection header received
0619 02:32:38.115497 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:38.115515 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:38.115870 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:38.116244 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:38.117163 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.118555 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.119483 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.120305 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.120505 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:38.124426 (+ 3921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:38.124435 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:38.124457 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:38.124516 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:38.127429 (+ 2913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:38.128125 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:38.128134 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:38.128142 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:38.128316 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:38.128715 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:38.128733 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:38.128736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:38.129359 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:38.130421 (+ 1062us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:38.130583 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 02:32:38.130913 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":153,"threads_started":1}
I20260619 02:32:38.132159 771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60800600ea9544229691af08477568a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60800600ea9544229691af08477568a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40869 } } }
I20260619 02:32:38.132434 771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:32:38.133543 775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:32:38.137058 775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:32:38.147265 775 catalog_manager.cc:1347] Generated new cluster ID: 7a0f18ae2bbc481c8f01dcb6384dee6e
I20260619 02:32:38.147454 775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:32:38.161303 775 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:32:38.162505 775 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:32:38.167284 775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2: Generated new TSK 0
I20260619 02:32:38.168048 775 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.96.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
I20260619 02:32:38.208992 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:40869
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:39763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:32:38.336035 790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:38.336333 790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:38.336380 790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:38.340546 790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:38.340629 790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:38.340691 790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:32:38.346285 790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:39763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:40869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:38.347399 790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:38.348423 790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:38.358446 798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:38.359403 796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:38.359409 795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:38.359449 790 server_base.cc:1034] running on GCE node
I20260619 02:32:38.359934 790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:38.360728 790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:38.361912 790 hybrid_clock.cc:648] HybridClock initialized: now 1781836358361891 us; error 33 us; skew 500 ppm
Jun 19 02:32:38 dist-test-slave-4qw6 krb5kdc[683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836358, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:38.364725 790 init.cc:377] Logged in from keytab as kudu/127.17.96.193@KRBTEST.COM (short username kudu)
I20260619 02:32:38.365723 790 webserver.cc:460] Webserver started at http://127.17.96.193:33085/ using document root <none> and password file <none>
I20260619 02:32:38.366369 790 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:38.366432 790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:38.366611 790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:32:38.368743 790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "7b8fd1a8b0894b4db316d191e61e0783"
format_stamp: "Formatted at 2026-06-19 02:32:38 on dist-test-slave-4qw6"
server_key: "c4f426952d1fce273e9814d60c83be5b"
server_key_iv: "fc9d5e0a3e142030e363f0b64ca0ae7a"
server_key_version: "encryptionkey@0"
I20260619 02:32:38.369307 790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "7b8fd1a8b0894b4db316d191e61e0783"
format_stamp: "Formatted at 2026-06-19 02:32:38 on dist-test-slave-4qw6"
server_key: "c4f426952d1fce273e9814d60c83be5b"
server_key_iv: "fc9d5e0a3e142030e363f0b64ca0ae7a"
server_key_version: "encryptionkey@0"
I20260619 02:32:38.373210 790 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 02:32:38.376139 805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:38.377446 790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:32:38.377595 790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "7b8fd1a8b0894b4db316d191e61e0783"
format_stamp: "Formatted at 2026-06-19 02:32:38 on dist-test-slave-4qw6"
server_key: "c4f426952d1fce273e9814d60c83be5b"
server_key_iv: "fc9d5e0a3e142030e363f0b64ca0ae7a"
server_key_version: "encryptionkey@0"
I20260619 02:32:38.377718 790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:38.388214 790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:38.391209 790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:38.391422 790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:38.392057 790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:32:38.392946 790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:32:38.392992 790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:38.393061 790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:32:38.393095 790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:38.402626 790 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:35917
I20260619 02:32:38.402652 918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:35917 every 8 connection(s)
I20260619 02:32:38.403605 790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:32:38.406713 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 790
I20260619 02:32:38.406854 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:32:38.407145 17795 external_mini_cluster.cc:1428] Setting key eede0cbf0735e40d14b23efc26a99471
Jun 19 02:32:38 dist-test-slave-4qw6 krb5kdc[683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836358, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.96.194@KRBTEST.COM; defaulting to no policy
I20260619 02:32:38.418561 767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:38.405512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:38605 (local address 127.17.96.254:40869)
0619 02:32:38.405668 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:38.405673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:38.406633 (+ 960us) server_negotiation.cc:407] Connection header received
0619 02:32:38.408148 (+ 1515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:38.408151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:38.408206 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:38.408319 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:38.409612 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.410233 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.411497 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.411758 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.411880 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:38.414655 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:38.414658 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:38.414660 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:38.414688 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:38.416159 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:38.416762 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:38.416766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:38.416772 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:38.416858 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:38.417270 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:38.417273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:38.417275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:38.417489 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:38.417599 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:38.418229 (+ 630us) server_negotiation.cc:299] Negotiation successful
0619 02:32:38.418376 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.194@KRBTEST.COM" created.
I20260619 02:32:38.419863 921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:38.406001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.193:38605)
0619 02:32:38.406468 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:38.406513 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:38.407897 (+ 1384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:38.408513 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:38.408521 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:38.408916 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:38.409446 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:38.409461 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.410400 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.410404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:38.411346 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:38.411356 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.411930 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.411934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:38.412128 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:38.412840 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:38.412871 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:38.414449 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:38.416295 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:38.416302 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:38.416321 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:38.416653 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:38.416969 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:38.416972 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:38.416974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:38.417128 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:38.417598 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:38.417607 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:38.417948 (+ 341us) client_negotiation.cc:769] Sending connection context
0619 02:32:38.418216 (+ 268us) client_negotiation.cc:240] Negotiation successful
0619 02:32:38.418636 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":113,"threads_started":1}
I20260619 02:32:38.421443 919 heartbeater.cc:347] Connected to a master server at 127.17.96.254:40869
I20260619 02:32:38.421729 919 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:38.422320 919 heartbeater.cc:511] Master 127.17.96.254:40869 requested a full tablet report, sending...
I20260619 02:32:38.424083 728 ts_manager.cc:194] Registered new tserver with Master: 7b8fd1a8b0894b4db316d191e61e0783 (127.17.96.193:35917)
I20260619 02:32:38.425275 728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:38605
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
I20260619 02:32:38.434653 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:40869
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:39763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:32:38.549578 924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:38.549968 924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:38.550019 924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:38.554121 924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:38.554199 924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:38.554268 924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:32:38.560009 924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:39763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:40869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:38.561245 924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:38.562332 924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:32:38.572554 924 server_base.cc:1034] running on GCE node
W20260619 02:32:38.572510 932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:38.572491 929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:38.572510 930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:38.573273 924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:38.574030 924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:38.575203 924 hybrid_clock.cc:648] HybridClock initialized: now 1781836358575183 us; error 32 us; skew 500 ppm
Jun 19 02:32:38 dist-test-slave-4qw6 krb5kdc[683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836358, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:38.578423 924 init.cc:377] Logged in from keytab as kudu/127.17.96.194@KRBTEST.COM (short username kudu)
I20260619 02:32:38.579461 924 webserver.cc:460] Webserver started at http://127.17.96.194:35503/ using document root <none> and password file <none>
I20260619 02:32:38.580076 924 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:38.580164 924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:38.580415 924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:32:38.582880 924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "41a83f0752ad4174925cac3c8e82faca"
format_stamp: "Formatted at 2026-06-19 02:32:38 on dist-test-slave-4qw6"
server_key: "0faab3067808fdde02eb390541b51e94"
server_key_iv: "cfa8b83f2d01a0aa8c80e803d91834c9"
server_key_version: "encryptionkey@0"
I20260619 02:32:38.583529 924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "41a83f0752ad4174925cac3c8e82faca"
format_stamp: "Formatted at 2026-06-19 02:32:38 on dist-test-slave-4qw6"
server_key: "0faab3067808fdde02eb390541b51e94"
server_key_iv: "cfa8b83f2d01a0aa8c80e803d91834c9"
server_key_version: "encryptionkey@0"
I20260619 02:32:38.587888 924 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:32:38.590699 939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:38.592027 924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:32:38.592170 924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "41a83f0752ad4174925cac3c8e82faca"
format_stamp: "Formatted at 2026-06-19 02:32:38 on dist-test-slave-4qw6"
server_key: "0faab3067808fdde02eb390541b51e94"
server_key_iv: "cfa8b83f2d01a0aa8c80e803d91834c9"
server_key_version: "encryptionkey@0"
I20260619 02:32:38.592298 924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:38.620503 924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:38.623669 924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:38.623891 924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:38.624524 924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:32:38.625397 924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:32:38.625445 924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:38.625540 924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:32:38.625576 924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:38.635226 924 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:41259
I20260619 02:32:38.635294 1052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:41259 every 8 connection(s)
I20260619 02:32:38.636175 924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:32:38.642035 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 924
I20260619 02:32:38.642153 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:32:38.642439 17795 external_mini_cluster.cc:1428] Setting key 2580992c5222d7f428c1132f6b9f34be
Jun 19 02:32:38 dist-test-slave-4qw6 krb5kdc[683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836358, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:38.651963 767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:38.638090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:34719 (local address 127.17.96.254:40869)
0619 02:32:38.638212 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:38.638217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:38.639130 (+ 913us) server_negotiation.cc:407] Connection header received
0619 02:32:38.640556 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:38.640560 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:38.640624 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:38.640728 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:38.642078 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.642749 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.644291 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.644524 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.644609 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:38.647688 (+ 3079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:38.647695 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:38.647702 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:38.647738 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:38.649621 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:38.650216 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:38.650223 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:38.650226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:38.650333 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:38.650678 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:38.650683 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:38.650687 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:38.650962 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:38.651068 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:38.651600 (+ 532us) server_negotiation.cc:299] Negotiation successful
0619 02:32:38.651751 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
WARNING: no policy specified for kudu/127.17.96.195@KRBTEST.COM; defaulting to no policy
I20260619 02:32:38.652729 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:38.638498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.194:34719)
0619 02:32:38.638972 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:38.639018 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:38.640324 (+ 1306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:38.640915 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:38.640922 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:38.641316 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:38.641903 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:38.641921 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.642905 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.642909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:38.644159 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:38.644172 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.644691 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.644698 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:38.644950 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:38.645640 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:38.645671 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:38.647487 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:38.649791 (+ 2304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:38.649803 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:38.649818 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:38.650108 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:38.650434 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:38.650436 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:38.650437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:38.650574 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:38.651069 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:38.651074 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:38.651330 (+ 256us) client_negotiation.cc:769] Sending connection context
0619 02:32:38.651491 (+ 161us) client_negotiation.cc:240] Negotiation successful
0619 02:32:38.651768 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":116,"threads_started":1}
I20260619 02:32:38.654050 1053 heartbeater.cc:347] Connected to a master server at 127.17.96.254:40869
I20260619 02:32:38.654316 1053 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:38.654927 1053 heartbeater.cc:511] Master 127.17.96.254:40869 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.195@KRBTEST.COM" created.
I20260619 02:32:38.656133 728 ts_manager.cc:194] Registered new tserver with Master: 41a83f0752ad4174925cac3c8e82faca (127.17.96.194:41259)
I20260619 02:32:38.656749 728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:34719
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
I20260619 02:32:38.672415 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:40869
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:39763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:32:38.792671 1058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:38.792997 1058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:38.793069 1058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:38.797328 1058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:38.797437 1058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:38.797537 1058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:32:38.803323 1058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:39763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:40869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:38.804517 1058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:38.805567 1058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:38.817826 1063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:38.817835 1064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:38.818394 1058 server_base.cc:1034] running on GCE node
W20260619 02:32:38.818426 1066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:38.818917 1058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:38.819628 1058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:38.820827 1058 hybrid_clock.cc:648] HybridClock initialized: now 1781836358820806 us; error 48 us; skew 500 ppm
Jun 19 02:32:38 dist-test-slave-4qw6 krb5kdc[683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836358, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:38.824041 1058 init.cc:377] Logged in from keytab as kudu/127.17.96.195@KRBTEST.COM (short username kudu)
I20260619 02:32:38.825104 1058 webserver.cc:460] Webserver started at http://127.17.96.195:46701/ using document root <none> and password file <none>
I20260619 02:32:38.825816 1058 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:38.825917 1058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:38.826162 1058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:32:38.828366 1058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "936121d434914b27a4cf51ad4157e40f"
format_stamp: "Formatted at 2026-06-19 02:32:38 on dist-test-slave-4qw6"
server_key: "f1ff164a8daf50f9188d293e435c79fb"
server_key_iv: "ad8b7532e6bccfc955570ff9f97b9299"
server_key_version: "encryptionkey@0"
I20260619 02:32:38.828966 1058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "936121d434914b27a4cf51ad4157e40f"
format_stamp: "Formatted at 2026-06-19 02:32:38 on dist-test-slave-4qw6"
server_key: "f1ff164a8daf50f9188d293e435c79fb"
server_key_iv: "ad8b7532e6bccfc955570ff9f97b9299"
server_key_version: "encryptionkey@0"
I20260619 02:32:38.833458 1058 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 02:32:38.836301 1073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:38.837913 1058 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:32:38.838065 1058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "936121d434914b27a4cf51ad4157e40f"
format_stamp: "Formatted at 2026-06-19 02:32:38 on dist-test-slave-4qw6"
server_key: "f1ff164a8daf50f9188d293e435c79fb"
server_key_iv: "ad8b7532e6bccfc955570ff9f97b9299"
server_key_version: "encryptionkey@0"
I20260619 02:32:38.838199 1058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:38.866958 1058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:38.870509 1058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:38.870719 1058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:38.871388 1058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:32:38.872267 1058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:32:38.872346 1058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:38.872427 1058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:32:38.872466 1058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:38.883661 1058 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:38251
I20260619 02:32:38.884610 1058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:32:38.885121 1186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:38251 every 8 connection(s)
I20260619 02:32:38.890166 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 1058
I20260619 02:32:38.890312 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:32:38.890662 17795 external_mini_cluster.cc:1428] Setting key dbd53c60a7857ad332a70314697653d1
Jun 19 02:32:38 dist-test-slave-4qw6 krb5kdc[683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836358, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:38.900930 767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:38.886951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:47415 (local address 127.17.96.254:40869)
0619 02:32:38.887119 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:38.887123 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:38.888193 (+ 1070us) server_negotiation.cc:407] Connection header received
0619 02:32:38.890283 (+ 2090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:38.890286 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:38.890355 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:38.890458 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:38.891942 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.892733 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.894160 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.894398 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.894514 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:38.897099 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:38.897103 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:38.897106 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:38.897140 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:38.898725 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:38.899317 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:38.899321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:38.899323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:38.899412 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:38.899769 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:38.899773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:38.899774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:38.899978 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:38.900084 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:38.900617 (+ 533us) server_negotiation.cc:299] Negotiation successful
0619 02:32:38.900750 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:32:38.901851 1189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:38.887390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.195:47415)
0619 02:32:38.887988 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:38.888063 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:38.890015 (+ 1952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:38.890648 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:38.890656 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:38.891085 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:38.891793 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:38.891819 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.892889 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.892894 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:38.894036 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:38.894047 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.894554 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.894558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:38.894747 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:38.895379 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:38.895397 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:38.896923 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:38.898865 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:38.898871 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:38.898909 (+ 38us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:38.899203 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:38.899510 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:38.899513 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:38.899515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:38.899667 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:38.900094 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:38.900103 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:38.900375 (+ 272us) client_negotiation.cc:769] Sending connection context
0619 02:32:38.900574 (+ 199us) client_negotiation.cc:240] Negotiation successful
0619 02:32:38.900927 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":115,"threads_started":1}
I20260619 02:32:38.903179 1187 heartbeater.cc:347] Connected to a master server at 127.17.96.254:40869
I20260619 02:32:38.903429 1187 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:38.904055 1187 heartbeater.cc:511] Master 127.17.96.254:40869 requested a full tablet report, sending...
I20260619 02:32:38.905329 728 ts_manager.cc:194] Registered new tserver with Master: 936121d434914b27a4cf51ad4157e40f (127.17.96.195:38251)
I20260619 02:32:38.906322 728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:47415
I20260619 02:32:38.917362 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:32:38.927256 767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:38.919580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60186 (local address 127.17.96.254:40869)
0619 02:32:38.919757 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:38.919764 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:38.919837 (+ 73us) server_negotiation.cc:407] Connection header received
0619 02:32:38.919979 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:38.919984 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:38.920053 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:38.920143 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:38.920933 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.921733 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.922586 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.922825 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.922911 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:38.924437 (+ 1526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:38.924440 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:38.924442 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:38.924460 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:38.925664 (+ 1204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:38.926129 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:38.926132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:38.926133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:38.926213 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:38.926502 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:38.926504 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:38.926505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:38.926721 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:38.926836 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:38.927026 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 02:32:38.927129 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 02:32:38.930332 728 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60186:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:32:38.933629 728 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:32:38.948616 1200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:38.942641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50816 (local address 127.17.96.193:35917)
0619 02:32:38.943974 (+ 1333us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:38.943981 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:38.944004 (+ 23us) server_negotiation.cc:407] Connection header received
0619 02:32:38.944080 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:38.944085 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:38.944289 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:38.944531 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:32:38.944768 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.946007 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.947330 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.947904 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.948047 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:38.948208 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:38.948347 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 02:32:38.948445 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1221,"thread_start_us":99,"threads_started":1}
I20260619 02:32:38.948601 1198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:38.942521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:35917 (local address 127.0.0.1:50816)
0619 02:32:38.943487 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:38.943540 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:38.943668 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:38.944441 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:38.944444 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:38.944462 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:32:38.944635 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:38.944640 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.946178 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.946181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:38.947214 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:38.947223 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.948043 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.948047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:38.948157 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:38.948196 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 02:32:38.948316 (+ 120us) client_negotiation.cc:240] Negotiation successful
0619 02:32:38.948449 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"mutex_wait_us":53,"thread_start_us":96,"threads_started":1}
I20260619 02:32:38.949168 1203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:38.943477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57530 (local address 127.17.96.194:41259)
0619 02:32:38.943961 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:38.943971 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:38.944223 (+ 252us) server_negotiation.cc:407] Connection header received
0619 02:32:38.944414 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:38.944420 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:38.944584 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:38.944722 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:32:38.944919 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.945636 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.947149 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.948034 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.948139 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:38.948262 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:38.948690 (+ 428us) server_negotiation.cc:299] Negotiation successful
0619 02:32:38.949010 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":147,"threads_started":1}
I20260619 02:32:38.949463 1199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:38.943052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:41259 (local address 127.0.0.1:57530)
0619 02:32:38.944104 (+ 1052us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:38.944119 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:38.944221 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:38.944704 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:38.944707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:38.944723 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:32:38.944824 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:38.944831 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.945792 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.945799 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:38.947025 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:38.947036 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.948323 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.948326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:38.948441 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:38.948464 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 02:32:38.948528 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 02:32:38.948611 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":993,"mutex_wait_us":3,"thread_start_us":98,"threads_started":1}
I20260619 02:32:38.950037 1202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:38.942610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:38251 (local address 127.0.0.1:53586)
0619 02:32:38.943504 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:38.943548 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:38.943709 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:38.944380 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:38.944385 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:38.944404 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:32:38.944623 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:38.944631 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.947190 (+ 2559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.947193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:38.948324 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:38.948331 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.949614 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.949617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:38.949709 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:38.949737 (+ 28us) client_negotiation.cc:769] Sending connection context
0619 02:32:38.949841 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 02:32:38.949916 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"mutex_wait_us":29,"thread_start_us":111,"threads_started":1}
I20260619 02:32:38.950503 1201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:38.942997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53586 (local address 127.17.96.195:38251)
0619 02:32:38.943362 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:38.943380 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:38.943630 (+ 250us) server_negotiation.cc:407] Connection header received
0619 02:32:38.943871 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:38.943877 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:38.944101 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:38.944291 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:32:38.944768 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.946006 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.948458 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:38.949305 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:38.949410 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:38.949550 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:38.950250 (+ 700us) server_negotiation.cc:299] Negotiation successful
0619 02:32:38.950366 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":132,"threads_started":1}
I20260619 02:32:38.952069 853 tablet_service.cc:1467] Processing CreateTablet for tablet c8964442039e4e00a9e7be30965d8b72 (DEFAULT_TABLE table=test-table [id=29c0ef2dd5da4e3fad848761a27fdaf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:32:38.952296 987 tablet_service.cc:1467] Processing CreateTablet for tablet c8964442039e4e00a9e7be30965d8b72 (DEFAULT_TABLE table=test-table [id=29c0ef2dd5da4e3fad848761a27fdaf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:32:38.953114 853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8964442039e4e00a9e7be30965d8b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:32:38.953279 987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8964442039e4e00a9e7be30965d8b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:32:38.953011 1121 tablet_service.cc:1467] Processing CreateTablet for tablet c8964442039e4e00a9e7be30965d8b72 (DEFAULT_TABLE table=test-table [id=29c0ef2dd5da4e3fad848761a27fdaf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:32:38.953972 1121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8964442039e4e00a9e7be30965d8b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:32:38.959661 1204 tablet_bootstrap.cc:492] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783: Bootstrap starting.
I20260619 02:32:38.961400 1205 tablet_bootstrap.cc:492] T c8964442039e4e00a9e7be30965d8b72 P 41a83f0752ad4174925cac3c8e82faca: Bootstrap starting.
I20260619 02:32:38.962077 1204 tablet_bootstrap.cc:654] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783: Neither blocks nor log segments found. Creating new log.
I20260619 02:32:38.962069 1206 tablet_bootstrap.cc:492] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f: Bootstrap starting.
I20260619 02:32:38.962946 1204 log.cc:824] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783: Log is configured to *not* fsync() on all Append() calls
I20260619 02:32:38.964120 1206 tablet_bootstrap.cc:654] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f: Neither blocks nor log segments found. Creating new log.
I20260619 02:32:38.964205 1205 tablet_bootstrap.cc:654] T c8964442039e4e00a9e7be30965d8b72 P 41a83f0752ad4174925cac3c8e82faca: Neither blocks nor log segments found. Creating new log.
I20260619 02:32:38.965034 1204 tablet_bootstrap.cc:492] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783: No bootstrap required, opened a new log
I20260619 02:32:38.965164 1205 log.cc:824] T c8964442039e4e00a9e7be30965d8b72 P 41a83f0752ad4174925cac3c8e82faca: Log is configured to *not* fsync() on all Append() calls
I20260619 02:32:38.965250 1204 ts_tablet_manager.cc:1397] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 02:32:38.965381 1206 log.cc:824] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:32:38.967155 1205 tablet_bootstrap.cc:492] T c8964442039e4e00a9e7be30965d8b72 P 41a83f0752ad4174925cac3c8e82faca: No bootstrap required, opened a new log
I20260619 02:32:38.967402 1205 ts_tablet_manager.cc:1397] T c8964442039e4e00a9e7be30965d8b72 P 41a83f0752ad4174925cac3c8e82faca: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 02:32:38.967447 1206 tablet_bootstrap.cc:492] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f: No bootstrap required, opened a new log
I20260619 02:32:38.967681 1206 ts_tablet_manager.cc:1397] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 02:32:38.970343 1204 raft_consensus.cc:348] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } } peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } }
I20260619 02:32:38.970618 1204 raft_consensus.cc:374] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:32:38.970690 1204 raft_consensus.cc:729] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b8fd1a8b0894b4db316d191e61e0783, State: Initialized, Role: FOLLOWER
I20260619 02:32:38.970964 1205 raft_consensus.cc:348] T c8964442039e4e00a9e7be30965d8b72 P 41a83f0752ad4174925cac3c8e82faca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } } peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } }
I20260619 02:32:38.971230 1205 raft_consensus.cc:374] T c8964442039e4e00a9e7be30965d8b72 P 41a83f0752ad4174925cac3c8e82faca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:32:38.971168 1204 consensus_queue.cc:260] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } } peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } }
I20260619 02:32:38.971323 1205 raft_consensus.cc:729] T c8964442039e4e00a9e7be30965d8b72 P 41a83f0752ad4174925cac3c8e82faca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41a83f0752ad4174925cac3c8e82faca, State: Initialized, Role: FOLLOWER
I20260619 02:32:38.971750 1205 consensus_queue.cc:260] T c8964442039e4e00a9e7be30965d8b72 P 41a83f0752ad4174925cac3c8e82faca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } } peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } }
I20260619 02:32:38.971897 919 heartbeater.cc:503] Master 127.17.96.254:40869 was elected leader, sending a full tablet report...
I20260619 02:32:38.972309 1204 ts_tablet_manager.cc:1428] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260619 02:32:38.972764 1205 ts_tablet_manager.cc:1428] T c8964442039e4e00a9e7be30965d8b72 P 41a83f0752ad4174925cac3c8e82faca: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 02:32:38.972966 1053 heartbeater.cc:503] Master 127.17.96.254:40869 was elected leader, sending a full tablet report...
I20260619 02:32:38.973037 1206 raft_consensus.cc:348] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } } peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } }
I20260619 02:32:38.973325 1206 raft_consensus.cc:374] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:32:38.973378 1206 raft_consensus.cc:729] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 936121d434914b27a4cf51ad4157e40f, State: Initialized, Role: FOLLOWER
I20260619 02:32:38.973802 1206 consensus_queue.cc:260] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } } peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } }
I20260619 02:32:38.974493 1187 heartbeater.cc:503] Master 127.17.96.254:40869 was elected leader, sending a full tablet report...
I20260619 02:32:38.975291 1206 ts_tablet_manager.cc:1428] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260619 02:32:39.010203 1212 raft_consensus.cc:482] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:32:39.010435 1212 raft_consensus.cc:504] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } } peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } }
I20260619 02:32:39.011660 1212 leader_election.cc:288] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b8fd1a8b0894b4db316d191e61e0783 (127.17.96.193:35917), 41a83f0752ad4174925cac3c8e82faca (127.17.96.194:41259)
I20260619 02:32:39.016158 1189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:39.011857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:35917 (local address 127.17.96.195:50349)
0619 02:32:39.012109 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:39.012127 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:39.012225 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:39.012582 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:39.012585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:39.012631 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:32:39.012750 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:39.012756 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:39.013500 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:39.013504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:39.014779 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:39.014791 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:39.015699 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:39.015702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:39.015781 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:39.015826 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 02:32:39.015905 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 02:32:39.015986 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":25,"spinlock_wait_cycles":9984}
I20260619 02:32:39.016189 1200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:39.012188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:50349 (local address 127.17.96.193:35917)
0619 02:32:39.012379 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:39.012382 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:39.012394 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:32:39.012440 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:39.012442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:39.012482 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:39.012570 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:32:39.012865 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:39.013382 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:39.014916 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:39.015569 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:39.015727 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:39.015805 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:39.015933 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 02:32:39.016028 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260619 02:32:39.016230 1213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:39.011965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:41259 (local address 127.17.96.195:57589)
0619 02:32:39.012455 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:39.012468 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:39.012636 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:39.012927 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:39.012930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:39.012947 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:32:39.013078 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:39.013084 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:39.013867 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:39.013871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:39.014998 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:39.015005 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:39.015845 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:39.015850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:39.015964 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:39.015984 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 02:32:39.016052 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 02:32:39.016104 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":27,"spinlock_wait_cycles":23168,"thread_start_us":148,"threads_started":1}
I20260619 02:32:39.016312 1203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:39.012235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:57589 (local address 127.17.96.194:41259)
0619 02:32:39.012383 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:39.012388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:39.012560 (+ 172us) server_negotiation.cc:407] Connection header received
0619 02:32:39.012758 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:39.012762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:39.012824 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:39.012918 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:32:39.013194 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:39.013701 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:39.015108 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:39.015739 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:39.015862 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:39.015942 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:39.016084 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 02:32:39.016156 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:32:39.016911 1007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8964442039e4e00a9e7be30965d8b72" candidate_uuid: "936121d434914b27a4cf51ad4157e40f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41a83f0752ad4174925cac3c8e82faca" is_pre_election: true
I20260619 02:32:39.017014 873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8964442039e4e00a9e7be30965d8b72" candidate_uuid: "936121d434914b27a4cf51ad4157e40f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b8fd1a8b0894b4db316d191e61e0783" is_pre_election: true
I20260619 02:32:39.017273 1007 raft_consensus.cc:2454] T c8964442039e4e00a9e7be30965d8b72 P 41a83f0752ad4174925cac3c8e82faca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 936121d434914b27a4cf51ad4157e40f in term 0.
I20260619 02:32:39.017336 873 raft_consensus.cc:2454] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 936121d434914b27a4cf51ad4157e40f in term 0.
I20260619 02:32:39.017907 1074 leader_election.cc:302] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 41a83f0752ad4174925cac3c8e82faca, 7b8fd1a8b0894b4db316d191e61e0783, 936121d434914b27a4cf51ad4157e40f; no voters:
I20260619 02:32:39.018242 1212 raft_consensus.cc:2781] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:32:39.018337 1212 raft_consensus.cc:482] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:32:39.018388 1212 raft_consensus.cc:3037] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:32:39.019585 1212 raft_consensus.cc:504] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } } peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } }
I20260619 02:32:39.019973 1212 leader_election.cc:288] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f [CANDIDATE]: Term 1 election: Requested vote from peers 7b8fd1a8b0894b4db316d191e61e0783 (127.17.96.193:35917), 41a83f0752ad4174925cac3c8e82faca (127.17.96.194:41259)
I20260619 02:32:39.020397 873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8964442039e4e00a9e7be30965d8b72" candidate_uuid: "936121d434914b27a4cf51ad4157e40f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b8fd1a8b0894b4db316d191e61e0783"
I20260619 02:32:39.020520 1007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8964442039e4e00a9e7be30965d8b72" candidate_uuid: "936121d434914b27a4cf51ad4157e40f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41a83f0752ad4174925cac3c8e82faca"
I20260619 02:32:39.020553 873 raft_consensus.cc:3037] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:32:39.020676 1007 raft_consensus.cc:3037] T c8964442039e4e00a9e7be30965d8b72 P 41a83f0752ad4174925cac3c8e82faca [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:32:39.021732 1007 raft_consensus.cc:2454] T c8964442039e4e00a9e7be30965d8b72 P 41a83f0752ad4174925cac3c8e82faca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 936121d434914b27a4cf51ad4157e40f in term 1.
I20260619 02:32:39.021806 873 raft_consensus.cc:2454] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 936121d434914b27a4cf51ad4157e40f in term 1.
I20260619 02:32:39.022173 1076 leader_election.cc:302] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 41a83f0752ad4174925cac3c8e82faca, 7b8fd1a8b0894b4db316d191e61e0783, 936121d434914b27a4cf51ad4157e40f; no voters:
I20260619 02:32:39.022383 1212 raft_consensus.cc:2781] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:32:39.022617 1212 raft_consensus.cc:686] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f [term 1 LEADER]: Becoming Leader. State: Replica: 936121d434914b27a4cf51ad4157e40f, State: Running, Role: LEADER
I20260619 02:32:39.022986 1212 consensus_queue.cc:237] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } } peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } }
I20260619 02:32:39.026407 728 catalog_manager.cc:5495] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f reported cstate change: term changed from 0 to 1, leader changed from <none> to 936121d434914b27a4cf51ad4157e40f (127.17.96.195). New cstate: current_term: 1 leader_uuid: "936121d434914b27a4cf51ad4157e40f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } health_report { overall_health: UNKNOWN } } }
I20260619 02:32:39.053511 1201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:39.050298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53594 (local address 127.17.96.195:38251)
0619 02:32:39.050431 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:39.050435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:39.050447 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:32:39.050524 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:39.050526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:39.050579 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:39.050672 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:32:39.050903 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:39.051402 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:39.052211 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:39.052418 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:39.052492 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:39.052751 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:32:39.053188 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:32:39.053282 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:39.053342 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 02:32:39.053393 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 02:32:39.059578 873 raft_consensus.cc:1261] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Refusing update from remote peer 936121d434914b27a4cf51ad4157e40f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:32:39.059579 1007 raft_consensus.cc:1261] T c8964442039e4e00a9e7be30965d8b72 P 41a83f0752ad4174925cac3c8e82faca [term 1 FOLLOWER]: Refusing update from remote peer 936121d434914b27a4cf51ad4157e40f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:32:39.060322 1212 consensus_queue.cc:1035] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f [LEADER]: Connected to new peer: Peer: permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:32:39.060576 1214 consensus_queue.cc:1035] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:32:39.070869 1219 mvcc.cc:205] Tried to move back new op lower bound from 7298401726700302336 to 7298401726561189888. Current Snapshot: MvccSnapshot[applied={T|T < 7298401726700302336 or (T in {7298401726700302336})}]
I20260619 02:32:39.071143 1218 mvcc.cc:205] Tried to move back new op lower bound from 7298401726700302336 to 7298401726561189888. Current Snapshot: MvccSnapshot[applied={T|T < 7298401726700302336}]
I20260619 02:32:39.077512 727 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:32:39.077730 727 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:32:39.080602 727 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2: Sending DeleteTablet for 3 replicas of tablet c8964442039e4e00a9e7be30965d8b72
I20260619 02:32:39.081571 1121 tablet_service.cc:1514] Processing DeleteTablet for tablet c8964442039e4e00a9e7be30965d8b72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:32:39 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:53586
I20260619 02:32:39.081699 853 tablet_service.cc:1514] Processing DeleteTablet for tablet c8964442039e4e00a9e7be30965d8b72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:32:39 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:50816
I20260619 02:32:39.081780 987 tablet_service.cc:1514] Processing DeleteTablet for tablet c8964442039e4e00a9e7be30965d8b72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:32:39 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:57530
I20260619 02:32:39.082489 1226 tablet_replica.cc:330] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f: stopping tablet replica
I20260619 02:32:39.082477 1228 tablet_replica.cc:330] T c8964442039e4e00a9e7be30965d8b72 P 41a83f0752ad4174925cac3c8e82faca: stopping tablet replica
I20260619 02:32:39.082528 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 790
I20260619 02:32:39.083284 1226 raft_consensus.cc:2229] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:32:39.083747 1226 raft_consensus.cc:2258] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:32:39.084515 1228 raft_consensus.cc:2229] T c8964442039e4e00a9e7be30965d8b72 P 41a83f0752ad4174925cac3c8e82faca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:32:39.084945 1228 raft_consensus.cc:2258] T c8964442039e4e00a9e7be30965d8b72 P 41a83f0752ad4174925cac3c8e82faca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:32:39.085677 1226 ts_tablet_manager.cc:1905] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:32:39.086440 1228 ts_tablet_manager.cc:1905] T c8964442039e4e00a9e7be30965d8b72 P 41a83f0752ad4174925cac3c8e82faca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:32:39.088698 1226 ts_tablet_manager.cc:1918] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:32:39.088824 1226 log.cc:1196] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/c8964442039e4e00a9e7be30965d8b72
I20260619 02:32:39.089181 1226 ts_tablet_manager.cc:1939] T c8964442039e4e00a9e7be30965d8b72 P 936121d434914b27a4cf51ad4157e40f: Deleting consensus metadata
I20260619 02:32:39.089550 1228 ts_tablet_manager.cc:1918] T c8964442039e4e00a9e7be30965d8b72 P 41a83f0752ad4174925cac3c8e82faca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:32:39.089718 1228 log.cc:1196] T c8964442039e4e00a9e7be30965d8b72 P 41a83f0752ad4174925cac3c8e82faca: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/c8964442039e4e00a9e7be30965d8b72
I20260619 02:32:39.090131 1228 ts_tablet_manager.cc:1939] T c8964442039e4e00a9e7be30965d8b72 P 41a83f0752ad4174925cac3c8e82faca: Deleting consensus metadata
I20260619 02:32:39.090830 713 catalog_manager.cc:4841] TS 936121d434914b27a4cf51ad4157e40f (127.17.96.195:38251): tablet c8964442039e4e00a9e7be30965d8b72 (table test-table [id=29c0ef2dd5da4e3fad848761a27fdaf8]) successfully deleted
I20260619 02:32:39.091418 714 catalog_manager.cc:4841] TS 41a83f0752ad4174925cac3c8e82faca (127.17.96.194:41259): tablet c8964442039e4e00a9e7be30965d8b72 (table test-table [id=29c0ef2dd5da4e3fad848761a27fdaf8]) successfully deleted
W20260619 02:32:39.091809 712 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.96.193:35917 (error 108)
I20260619 02:32:39.092079 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 924
I20260619 02:32:39.093246 1202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:39.092978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:35917 (local address 127.0.0.1:50830)
0619 02:32:39.093103 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:39.093172 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.193:35917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260619 02:32:39.093468 712 catalog_manager.cc:4584] TS 7b8fd1a8b0894b4db316d191e61e0783 (127.17.96.193:35917): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c8964442039e4e00a9e7be30965d8b72: Network error: Client connection negotiation failed: client connection to 127.17.96.193:35917: connect: Connection refused (error 111)
I20260619 02:32:39.099095 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 1058
I20260619 02:32:39.104909 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 696
I20260619 02:32:59.111490 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:40869
--webserver_interface=127.17.96.254
--webserver_port=44899
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:39763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:40869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:32:59.224753 1230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:59.225064 1230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:59.225107 1230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:59.229334 1230 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 02:32:59.229411 1230 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 02:32:59.229429 1230 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 02:32:59.229444 1230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:32:59.229462 1230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:59.229480 1230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:59.229496 1230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:32:59.229548 1230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:32:59.235167 1230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:39763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:40869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:40869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=44899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:59.236377 1230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:59.237471 1230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:59.251118 1236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:59.251129 1237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:59.251194 1230 server_base.cc:1034] running on GCE node
W20260619 02:32:59.251153 1239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:59.252066 1230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:59.253310 1230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:59.254510 1230 hybrid_clock.cc:648] HybridClock initialized: now 1781836379254456 us; error 70 us; skew 500 ppm
Jun 19 02:32:59 dist-test-slave-4qw6 krb5kdc[683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836379, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:59.257422 1230 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:32:59.258491 1230 webserver.cc:460] Webserver started at http://127.17.96.254:44899/ using document root <none> and password file <none>
I20260619 02:32:59.259141 1230 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:59.259261 1230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:59.263507 1230 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:32:59.265829 1246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:59.267165 1230 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 02:32:59.267324 1230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "60800600ea9544229691af08477568a2"
format_stamp: "Formatted at 2026-06-19 02:32:38 on dist-test-slave-4qw6"
server_key: "525651ac22f1f7824b16f1c30cf38818"
server_key_iv: "766d3624adb6bf407434b06fab1d4221"
server_key_version: "encryptionkey@0"
I20260619 02:32:59.267982 1230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:59.289062 1230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:59.292246 1230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:59.292479 1230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:59.300253 1230 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:40869
I20260619 02:32:59.300546 1298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:40869 every 8 connection(s)
I20260619 02:32:59.301491 1230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:32:59.306237 1299 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:32:59.308082 1299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2: Bootstrap starting.
I20260619 02:32:59.309267 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 1230
I20260619 02:32:59.309850 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:35917
--local_ip_for_outbound_sockets=127.17.96.193
--tserver_master_addrs=127.17.96.254:40869
--webserver_port=33085
--webserver_interface=127.17.96.193
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:39763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:32:59.327065 1299 log.cc:824] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2: Log is configured to *not* fsync() on all Append() calls
I20260619 02:32:59.336334 1299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:32:59.336796 1299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2: Bootstrap complete.
I20260619 02:32:59.341143 1299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60800600ea9544229691af08477568a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40869 } }
I20260619 02:32:59.341573 1299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60800600ea9544229691af08477568a2, State: Initialized, Role: FOLLOWER
I20260619 02:32:59.342114 1299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60800600ea9544229691af08477568a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40869 } }
I20260619 02:32:59.342238 1299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:32:59.342334 1299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:32:59.342456 1299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:32:59.344575 1299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60800600ea9544229691af08477568a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40869 } }
I20260619 02:32:59.344950 1299 leader_election.cc:302] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60800600ea9544229691af08477568a2; no voters:
I20260619 02:32:59.345281 1299 leader_election.cc:288] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 02:32:59.345438 1303 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 02:32:59.345706 1303 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 2 LEADER]: Becoming Leader. State: Replica: 60800600ea9544229691af08477568a2, State: Running, Role: LEADER
I20260619 02:32:59.346123 1303 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60800600ea9544229691af08477568a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40869 } }
I20260619 02:32:59.346634 1299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:32:59.347687 1305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60800600ea9544229691af08477568a2. Latest consensus state: current_term: 2 leader_uuid: "60800600ea9544229691af08477568a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60800600ea9544229691af08477568a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40869 } } }
I20260619 02:32:59.347841 1305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:32:59.348090 1304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "60800600ea9544229691af08477568a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60800600ea9544229691af08477568a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40869 } } }
I20260619 02:32:59.348168 1304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:32:59.348217 1312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:32:59.353147 1312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:32:59.353828 1312 catalog_manager.cc:1259] Loaded cluster ID: 7a0f18ae2bbc481c8f01dcb6384dee6e
I20260619 02:32:59.353890 1312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:32:59.356599 1312 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:32:59.357533 1312 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2: Loaded TSK: 0
I20260619 02:32:59.375097 1312 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2: Generated new TSK 1
I20260619 02:32:59.375427 1312 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:32:59.434480 1301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:59.434764 1301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:59.434810 1301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:59.439085 1301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:59.439173 1301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:59.439240 1301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:32:59.444996 1301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:39763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:35917
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=33085
--tserver_master_addrs=127.17.96.254:40869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:59.446262 1301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:59.447402 1301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:59.457650 1326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:59.457772 1329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:59.457695 1327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:59.458165 1301 server_base.cc:1034] running on GCE node
I20260619 02:32:59.458719 1301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:59.459622 1301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:59.460843 1301 hybrid_clock.cc:648] HybridClock initialized: now 1781836379460819 us; error 41 us; skew 500 ppm
Jun 19 02:32:59 dist-test-slave-4qw6 krb5kdc[683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836379, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:59.464563 1301 init.cc:377] Logged in from keytab as kudu/127.17.96.193@KRBTEST.COM (short username kudu)
I20260619 02:32:59.465919 1301 webserver.cc:460] Webserver started at http://127.17.96.193:33085/ using document root <none> and password file <none>
I20260619 02:32:59.466722 1301 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:59.466876 1301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:59.472549 1301 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260619 02:32:59.475451 1336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:59.477064 1301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:32:59.477228 1301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "7b8fd1a8b0894b4db316d191e61e0783"
format_stamp: "Formatted at 2026-06-19 02:32:38 on dist-test-slave-4qw6"
server_key: "c4f426952d1fce273e9814d60c83be5b"
server_key_iv: "fc9d5e0a3e142030e363f0b64ca0ae7a"
server_key_version: "encryptionkey@0"
I20260619 02:32:59.478034 1301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:59.509905 1301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:59.514041 1301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:59.514238 1301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:59.514859 1301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:32:59.516007 1343 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 02:32:59.520344 1301 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 02:32:59.520428 1301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260619 02:32:59.520587 1301 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 02:32:59.522576 1301 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 02:32:59.522639 1301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260619 02:32:59.522974 1343 tablet_bootstrap.cc:492] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783: Bootstrap starting.
I20260619 02:32:59.534216 1301 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:35917
I20260619 02:32:59.534251 1450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:35917 every 8 connection(s)
I20260619 02:32:59.535296 1301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:32:59.538268 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 1301
I20260619 02:32:59.538760 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:41259
--local_ip_for_outbound_sockets=127.17.96.194
--tserver_master_addrs=127.17.96.254:40869
--webserver_port=35503
--webserver_interface=127.17.96.194
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:39763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:32:59.544325 1343 log.cc:824] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783: Log is configured to *not* fsync() on all Append() calls
Jun 19 02:32:59 dist-test-slave-4qw6 krb5kdc[683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836379, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:59.553257 1343 tablet_bootstrap.cc:492] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:32:59.553689 1343 tablet_bootstrap.cc:492] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783: Bootstrap complete.
I20260619 02:32:59.554958 1343 ts_tablet_manager.cc:1397] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783: Time spent bootstrapping tablet: real 0.032s user 0.026s sys 0.004s
I20260619 02:32:59.557068 1453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:59.537788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:43397 (local address 127.17.96.254:40869)
0619 02:32:59.540371 (+ 2583us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:59.540401 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:59.540460 (+ 59us) server_negotiation.cc:407] Connection header received
0619 02:32:59.540817 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:59.540832 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:59.541292 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:59.541733 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:59.543303 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.544807 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.546412 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.547420 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.547643 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:59.550969 (+ 3326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:59.550978 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:59.550997 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:59.551046 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:59.553258 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:59.554099 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:59.554105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:59.554110 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:59.554283 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:59.554711 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:59.554715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:59.554716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:59.555179 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:59.555321 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:59.555945 (+ 624us) server_negotiation.cc:299] Negotiation successful
0619 02:32:59.556261 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2374,"thread_start_us":119,"threads_started":1}
I20260619 02:32:59.557091 1454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:59.537737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.193:43397)
0619 02:32:59.538471 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:59.538536 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:59.539765 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:59.541809 (+ 2044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:59.541822 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:59.542376 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:59.543062 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:59.543086 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.544995 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.544999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:59.546260 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:59.546271 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.547592 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.547596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:59.547777 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:59.548547 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:59.548569 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:59.550665 (+ 2096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:59.553417 (+ 2752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:59.553423 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:59.553437 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:59.553944 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:59.554398 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:59.554404 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:59.554409 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:59.554595 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:59.555311 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:59.555318 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:59.555601 (+ 283us) client_negotiation.cc:769] Sending connection context
0619 02:32:59.555794 (+ 193us) client_negotiation.cc:240] Negotiation successful
0619 02:32:59.556141 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":177,"threads_started":1}
I20260619 02:32:59.557982 1343 raft_consensus.cc:348] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } } peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } }
I20260619 02:32:59.558485 1343 raft_consensus.cc:729] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b8fd1a8b0894b4db316d191e61e0783, State: Initialized, Role: FOLLOWER
I20260619 02:32:59.559029 1343 consensus_queue.cc:260] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } } peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } }
I20260619 02:32:59.560115 1343 ts_tablet_manager.cc:1428] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 02:32:59.560171 1451 heartbeater.cc:347] Connected to a master server at 127.17.96.254:40869
I20260619 02:32:59.560387 1451 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:59.560837 1451 heartbeater.cc:511] Master 127.17.96.254:40869 requested a full tablet report, sending...
I20260619 02:32:59.563328 1263 ts_manager.cc:194] Registered new tserver with Master: 7b8fd1a8b0894b4db316d191e61e0783 (127.17.96.193:35917)
I20260619 02:32:59.567353 1263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:43397
I20260619 02:32:59.570256 1451 heartbeater.cc:503] Master 127.17.96.254:40869 was elected leader, sending a full tablet report...
Jun 19 02:32:59 dist-test-slave-4qw6 krb5kdc[683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836379, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for kudu/127.17.96.193@KRBTEST.COM
I20260619 02:32:59.581203 1458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:59.566297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:35917 (local address 127.0.0.1:54516)
0619 02:32:59.566761 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:59.566812 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:59.566952 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:59.567681 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:59.567687 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:59.568126 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:59.568290 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:59.568299 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.569663 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.569667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:59.570721 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:59.570734 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.571494 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.571498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:59.571611 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:59.575465 (+ 3854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:59.575481 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:59.577081 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:59.579376 (+ 2295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:59.579385 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:59.579390 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:59.579658 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:59.579989 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:59.579991 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:59.579992 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:59.580068 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:59.580635 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:59.580640 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:59.580751 (+ 111us) client_negotiation.cc:769] Sending connection context
0619 02:32:59.580864 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 02:32:59.581032 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":146,"threads_started":1}
I20260619 02:32:59.581357 1459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:59.566436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54516 (local address 127.17.96.193:35917)
0619 02:32:59.567117 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:59.567128 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:59.567152 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:32:59.567225 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:59.567230 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:59.567435 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:59.567593 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:59.568416 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.569515 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.570840 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.571375 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.571486 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:59.577266 (+ 5780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:59.577280 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:59.577294 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:59.577324 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:59.579242 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:59.579779 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:59.579787 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:59.579789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:59.579893 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:59.580157 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:59.580160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:59.580162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:59.580513 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:59.580649 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:59.580971 (+ 322us) server_negotiation.cc:299] Negotiation successful
0619 02:32:59.581162 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":586,"thread_start_us":100,"threads_started":1}
I20260619 02:32:59.582584 1385 tablet_service.cc:1514] Processing DeleteTablet for tablet c8964442039e4e00a9e7be30965d8b72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:32:39 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:54516
I20260619 02:32:59.583653 1460 tablet_replica.cc:330] stopping tablet replica
I20260619 02:32:59.584007 1460 raft_consensus.cc:2229] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:32:59.584314 1460 raft_consensus.cc:2258] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:32:59.585345 1460 ts_tablet_manager.cc:1905] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:32:59.589232 1460 ts_tablet_manager.cc:1918] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:32:59.589362 1460 log.cc:1196] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/wal/wals/c8964442039e4e00a9e7be30965d8b72
I20260619 02:32:59.589636 1460 ts_tablet_manager.cc:1939] T c8964442039e4e00a9e7be30965d8b72 P 7b8fd1a8b0894b4db316d191e61e0783: Deleting consensus metadata
I20260619 02:32:59.591563 1247 catalog_manager.cc:4841] TS 7b8fd1a8b0894b4db316d191e61e0783 (127.17.96.193:35917): tablet c8964442039e4e00a9e7be30965d8b72 (table test-table [id=29c0ef2dd5da4e3fad848761a27fdaf8]) successfully deleted
W20260619 02:32:59.665512 1455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:59.665903 1455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:59.665956 1455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:59.670171 1455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:59.670248 1455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:59.670315 1455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:32:59.676189 1455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:39763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:41259
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=35503
--tserver_master_addrs=127.17.96.254:40869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:59.677297 1455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:59.678340 1455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:59.688330 1465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:59.688410 1455 server_base.cc:1034] running on GCE node
W20260619 02:32:59.688330 1466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:59.688330 1468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:59.689100 1455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:59.689818 1455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:59.691061 1455 hybrid_clock.cc:648] HybridClock initialized: now 1781836379691038 us; error 35 us; skew 500 ppm
Jun 19 02:32:59 dist-test-slave-4qw6 krb5kdc[683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836379, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:59.694007 1455 init.cc:377] Logged in from keytab as kudu/127.17.96.194@KRBTEST.COM (short username kudu)
I20260619 02:32:59.694964 1455 webserver.cc:460] Webserver started at http://127.17.96.194:35503/ using document root <none> and password file <none>
I20260619 02:32:59.695580 1455 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:59.695699 1455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:59.699836 1455 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:32:59.702078 1475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:59.703353 1455 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260619 02:32:59.703518 1455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "41a83f0752ad4174925cac3c8e82faca"
format_stamp: "Formatted at 2026-06-19 02:32:38 on dist-test-slave-4qw6"
server_key: "0faab3067808fdde02eb390541b51e94"
server_key_iv: "cfa8b83f2d01a0aa8c80e803d91834c9"
server_key_version: "encryptionkey@0"
I20260619 02:32:59.704142 1455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:59.724368 1455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:59.727669 1455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:59.727901 1455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:59.728561 1455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:32:59.729444 1455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:32:59.729528 1455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:59.729596 1455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:32:59.729640 1455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:59.739015 1455 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:41259
I20260619 02:32:59.739037 1588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:41259 every 8 connection(s)
I20260619 02:32:59.740056 1455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:32:59.747203 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 1455
I20260619 02:32:59.747730 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:38251
--local_ip_for_outbound_sockets=127.17.96.195
--tserver_master_addrs=127.17.96.254:40869
--webserver_port=46701
--webserver_interface=127.17.96.195
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:39763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:32:59 dist-test-slave-4qw6 krb5kdc[683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836379, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:59.755316 1453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:59.741935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:44591 (local address 127.17.96.254:40869)
0619 02:32:59.742101 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:59.742106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:59.743026 (+ 920us) server_negotiation.cc:407] Connection header received
0619 02:32:59.744330 (+ 1304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:59.744332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:59.744380 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:59.744456 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:59.745649 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.746199 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.747600 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.748047 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.748176 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:59.751385 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:59.751389 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:59.751391 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:59.751420 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:59.753012 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:59.753610 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:59.753613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:59.753614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:59.753704 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:59.754111 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:59.754116 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:59.754118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:59.754322 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:59.754436 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:59.754986 (+ 550us) server_negotiation.cc:299] Negotiation successful
0619 02:32:59.755142 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 02:32:59.756211 1591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:59.742413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.194:44591)
0619 02:32:59.742862 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:59.742906 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:59.744132 (+ 1226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:59.744624 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:59.744630 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:59.744999 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:59.745508 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:59.745521 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.746352 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.746355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:59.747459 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:59.747471 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.748244 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.748248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:59.748418 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:59.749218 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:59.749246 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:59.751161 (+ 1915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:59.753158 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:59.753165 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:59.753184 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:59.753480 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:59.753841 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:59.753845 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:59.753847 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:59.754005 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:59.754435 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:59.754444 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:59.754728 (+ 284us) client_negotiation.cc:769] Sending connection context
0619 02:32:59.754930 (+ 202us) client_negotiation.cc:240] Negotiation successful
0619 02:32:59.755308 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":113,"threads_started":1}
I20260619 02:32:59.757532 1589 heartbeater.cc:347] Connected to a master server at 127.17.96.254:40869
I20260619 02:32:59.757862 1589 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:59.758400 1589 heartbeater.cc:511] Master 127.17.96.254:40869 requested a full tablet report, sending...
I20260619 02:32:59.759655 1263 ts_manager.cc:194] Registered new tserver with Master: 41a83f0752ad4174925cac3c8e82faca (127.17.96.194:41259)
I20260619 02:32:59.760537 1263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:44591
W20260619 02:32:59.866859 1592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:32:59.867165 1592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:32:59.867213 1592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:32:59.871387 1592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:32:59.871456 1592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:32:59.871523 1592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:32:59.877351 1592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:39763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:38251
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=46701
--tserver_master_addrs=127.17.96.254:40869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:32:59.878557 1592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:32:59.879544 1592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:32:59.889921 1597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:59.890046 1592 server_base.cc:1034] running on GCE node
W20260619 02:32:59.889921 1598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:32:59.889941 1600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:32:59.890601 1592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:32:59.891325 1592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:32:59.892522 1592 hybrid_clock.cc:648] HybridClock initialized: now 1781836379892491 us; error 50 us; skew 500 ppm
Jun 19 02:32:59 dist-test-slave-4qw6 krb5kdc[683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836379, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:32:59.895422 1592 init.cc:377] Logged in from keytab as kudu/127.17.96.195@KRBTEST.COM (short username kudu)
I20260619 02:32:59.896420 1592 webserver.cc:460] Webserver started at http://127.17.96.195:46701/ using document root <none> and password file <none>
I20260619 02:32:59.897046 1592 fs_manager.cc:359] Metadata directory not provided
I20260619 02:32:59.897167 1592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:32:59.901183 1592 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:32:59.903666 1607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:59.904942 1592 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 02:32:59.905100 1592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "936121d434914b27a4cf51ad4157e40f"
format_stamp: "Formatted at 2026-06-19 02:32:38 on dist-test-slave-4qw6"
server_key: "f1ff164a8daf50f9188d293e435c79fb"
server_key_iv: "ad8b7532e6bccfc955570ff9f97b9299"
server_key_version: "encryptionkey@0"
I20260619 02:32:59.905678 1592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:32:59.927529 1592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:32:59.931164 1592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:32:59.931406 1592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:32:59.932044 1592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:32:59.932914 1592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:32:59.932963 1592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:59.933037 1592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:32:59.933073 1592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:32:59.942173 1592 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:38251
I20260619 02:32:59.942194 1720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:38251 every 8 connection(s)
I20260619 02:32:59.943289 1592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:32:59.945582 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 1592
Jun 19 02:32:59 dist-test-slave-4qw6 krb5kdc[683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836379, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:32:59.955150 1725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:59.946356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34478 (local address 127.17.96.254:40869)
0619 02:32:59.946604 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:59.946608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:59.946773 (+ 165us) server_negotiation.cc:407] Connection header received
0619 02:32:59.946887 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:59.946892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:59.946959 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:59.947071 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:59.947897 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.948749 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.949636 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.949991 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.950073 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:59.951590 (+ 1517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:59.951594 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:59.951598 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:59.951626 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:59.953435 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:59.953933 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:59.953936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:59.953937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:59.954014 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:59.954303 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:59.954305 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:59.954306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:59.954505 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:59.954698 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:59.954874 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 02:32:59.954978 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":88,"threads_started":1}
I20260619 02:32:59.958798 1453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:59.945202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:59185 (local address 127.17.96.254:40869)
0619 02:32:59.945379 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:59.945385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:59.946411 (+ 1026us) server_negotiation.cc:407] Connection header received
0619 02:32:59.948290 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:59.948295 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:59.948351 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:59.948412 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:59.949837 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.950498 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.951795 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.952116 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.952231 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:59.955167 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:59.955171 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:59.955175 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:59.955207 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:59.956650 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:59.957188 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:59.957192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:59.957194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:59.957273 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:59.957627 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:59.957630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:59.957632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:59.957872 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:59.957971 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:59.958497 (+ 526us) server_negotiation.cc:299] Negotiation successful
0619 02:32:59.958615 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 02:32:59.959661 1723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:59.945655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.195:59185)
0619 02:32:59.946242 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:59.946304 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:59.948005 (+ 1701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:59.948616 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:59.948625 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:59.949080 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:32:59.949657 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:59.949676 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.950634 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.950641 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:59.951671 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:59.951684 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.952309 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.952315 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:59.952561 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:59.953067 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:32:59.953089 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:32:59.954988 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:32:59.956786 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:59.956791 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:59.956808 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:59.957081 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:59.957363 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:32:59.957367 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:32:59.957369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:32:59.957524 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:32:59.957981 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:32:59.957990 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:32:59.958256 (+ 266us) client_negotiation.cc:769] Sending connection context
0619 02:32:59.958444 (+ 188us) client_negotiation.cc:240] Negotiation successful
0619 02:32:59.958798 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":121,"threads_started":1}
I20260619 02:32:59.960796 1721 heartbeater.cc:347] Connected to a master server at 127.17.96.254:40869
I20260619 02:32:59.961015 1721 heartbeater.cc:464] Registering TS with master...
I20260619 02:32:59.961465 1721 heartbeater.cc:511] Master 127.17.96.254:40869 requested a full tablet report, sending...
I20260619 02:32:59.962755 1263 ts_manager.cc:194] Registered new tserver with Master: 936121d434914b27a4cf51ad4157e40f (127.17.96.195:38251)
I20260619 02:32:59.963335 1263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:59185
I20260619 02:32:59.968755 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:32:59.972937 1453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:59.969450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34492 (local address 127.17.96.254:40869)
0619 02:32:59.969576 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:59.969580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:59.969885 (+ 305us) server_negotiation.cc:407] Connection header received
0619 02:32:59.970041 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:59.970043 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:59.970147 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:59.970230 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:32:59.970451 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.970961 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.971826 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.972039 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.972118 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:59.972359 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:32:59.972503 (+ 144us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0619 02:32:59.972800 (+ 297us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34492: token expired
Metrics: {"server-negotiator.queue_time_us":50}
W20260619 02:32:59.973019 1453 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34492: token expired
W20260619 02:32:59.973414 1191 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.17.96.254:40869)
I20260619 02:32:59.981312 1453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:59.973933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34496 (local address 127.17.96.254:40869)
0619 02:32:59.974035 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:59.974039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:59.974145 (+ 106us) server_negotiation.cc:407] Connection header received
0619 02:32:59.974283 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:59.974285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:59.974321 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:59.974383 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:32:59.975331 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.975884 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.976692 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.976911 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.977018 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:32:59.978399 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:32:59.978403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:32:59.978406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:32:59.978437 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:32:59.979681 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:59.980105 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:59.980110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:59.980112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:59.980194 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:32:59.980463 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:32:59.980468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:32:59.980469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:32:59.980662 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:32:59.980802 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 02:32:59.981029 (+ 227us) server_negotiation.cc:299] Negotiation successful
0619 02:32:59.981146 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 02:32:59.984015 1263 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:32:59.987475 1263 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:32:59.995357 1385 tablet_service.cc:1467] Processing CreateTablet for tablet 2fa3c1e985db4fe68da721583d4eed69 (DEFAULT_TABLE table=test-table [id=b0e5d79f24bf4a21a1f6c0dd2c89d1de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:32:59.996114 1385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fa3c1e985db4fe68da721583d4eed69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:32:59.999282 1343 tablet_bootstrap.cc:492] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783: Bootstrap starting.
I20260619 02:33:00.000272 1343 tablet_bootstrap.cc:654] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:00.000329 1731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:59.994503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:41259 (local address 127.0.0.1:35630)
0619 02:32:59.994844 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:59.994858 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:59.994945 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:59.995817 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:59.995820 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:59.995839 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:32:59.995953 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:59.995959 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.997696 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.997702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:59.998702 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:59.998712 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.999848 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.999852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:59.999946 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:00.000001 (+ 55us) client_negotiation.cc:769] Sending connection context
0619 02:33:00.000077 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 02:33:00.000164 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":121,"thread_start_us":109,"threads_started":1}
I20260619 02:33:00.000422 1732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:59.994913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35630 (local address 127.17.96.194:41259)
0619 02:32:59.995330 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:59.995346 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:59.995363 (+ 17us) server_negotiation.cc:407] Connection header received
0619 02:32:59.995424 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:59.995428 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:59.995631 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:59.995802 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:32:59.996083 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.997432 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.998837 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.999595 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.999721 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:00.000041 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:00.000126 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 02:33:00.000249 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":164,"threads_started":1}
I20260619 02:33:00.001272 1458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:59.994251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:38251 (local address 127.0.0.1:35300)
0619 02:32:59.994445 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 02:32:59.994461 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:32:59.994553 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:32:59.996033 (+ 1480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:32:59.996038 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:32:59.996061 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:32:59.996171 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:59.996177 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.997701 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.997704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:32:59.999044 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:32:59.999054 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:00.000871 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:00.000875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:00.000996 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:00.001020 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 02:33:00.001070 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 02:33:00.001150 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":52}
I20260619 02:33:00.002097 1343 tablet_bootstrap.cc:492] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783: No bootstrap required, opened a new log
I20260619 02:33:00.002102 1730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:32:59.994415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35300 (local address 127.17.96.195:38251)
0619 02:32:59.995251 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0619 02:32:59.995268 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 02:32:59.995299 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:32:59.995392 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:32:59.995398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:32:59.995631 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:32:59.995802 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:32:59.996316 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:32:59.997606 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:32:59.999505 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:00.000732 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:00.001568 (+ 836us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:00.001742 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:00.001849 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 02:33:00.001958 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":152,"threads_started":1}
I20260619 02:33:00.002206 1343 ts_tablet_manager.cc:1397] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 02:33:00.002743 1343 raft_consensus.cc:348] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } } peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } }
I20260619 02:33:00.002848 1343 raft_consensus.cc:374] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:00.002875 1343 raft_consensus.cc:729] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b8fd1a8b0894b4db316d191e61e0783, State: Initialized, Role: FOLLOWER
I20260619 02:33:00.003033 1343 consensus_queue.cc:260] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } } peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } }
I20260619 02:33:00.003410 1343 ts_tablet_manager.cc:1428] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 02:33:00.003995 1523 tablet_service.cc:1467] Processing CreateTablet for tablet 2fa3c1e985db4fe68da721583d4eed69 (DEFAULT_TABLE table=test-table [id=b0e5d79f24bf4a21a1f6c0dd2c89d1de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:33:00.005251 1523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fa3c1e985db4fe68da721583d4eed69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:00.004943 1655 tablet_service.cc:1467] Processing CreateTablet for tablet 2fa3c1e985db4fe68da721583d4eed69 (DEFAULT_TABLE table=test-table [id=b0e5d79f24bf4a21a1f6c0dd2c89d1de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:33:00.005952 1655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fa3c1e985db4fe68da721583d4eed69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:00.012862 1735 tablet_bootstrap.cc:492] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca: Bootstrap starting.
I20260619 02:33:00.012862 1734 tablet_bootstrap.cc:492] T 2fa3c1e985db4fe68da721583d4eed69 P 936121d434914b27a4cf51ad4157e40f: Bootstrap starting.
I20260619 02:33:00.015218 1734 tablet_bootstrap.cc:654] T 2fa3c1e985db4fe68da721583d4eed69 P 936121d434914b27a4cf51ad4157e40f: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:00.015631 1735 tablet_bootstrap.cc:654] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:00.016350 1735 log.cc:824] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:00.016351 1734 log.cc:824] T 2fa3c1e985db4fe68da721583d4eed69 P 936121d434914b27a4cf51ad4157e40f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:00.018376 1735 tablet_bootstrap.cc:492] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca: No bootstrap required, opened a new log
I20260619 02:33:00.018378 1734 tablet_bootstrap.cc:492] T 2fa3c1e985db4fe68da721583d4eed69 P 936121d434914b27a4cf51ad4157e40f: No bootstrap required, opened a new log
I20260619 02:33:00.018595 1735 ts_tablet_manager.cc:1397] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 02:33:00.018589 1734 ts_tablet_manager.cc:1397] T 2fa3c1e985db4fe68da721583d4eed69 P 936121d434914b27a4cf51ad4157e40f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 02:33:00.021895 1735 raft_consensus.cc:348] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } } peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } }
I20260619 02:33:00.021896 1734 raft_consensus.cc:348] T 2fa3c1e985db4fe68da721583d4eed69 P 936121d434914b27a4cf51ad4157e40f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } } peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } }
I20260619 02:33:00.022177 1735 raft_consensus.cc:374] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:00.022200 1734 raft_consensus.cc:374] T 2fa3c1e985db4fe68da721583d4eed69 P 936121d434914b27a4cf51ad4157e40f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:00.022260 1734 raft_consensus.cc:729] T 2fa3c1e985db4fe68da721583d4eed69 P 936121d434914b27a4cf51ad4157e40f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 936121d434914b27a4cf51ad4157e40f, State: Initialized, Role: FOLLOWER
I20260619 02:33:00.022269 1735 raft_consensus.cc:729] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41a83f0752ad4174925cac3c8e82faca, State: Initialized, Role: FOLLOWER
I20260619 02:33:00.022735 1735 consensus_queue.cc:260] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } } peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } }
I20260619 02:33:00.022734 1734 consensus_queue.cc:260] T 2fa3c1e985db4fe68da721583d4eed69 P 936121d434914b27a4cf51ad4157e40f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } } peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } }
I20260619 02:33:00.023438 1721 heartbeater.cc:503] Master 127.17.96.254:40869 was elected leader, sending a full tablet report...
I20260619 02:33:00.023438 1589 heartbeater.cc:503] Master 127.17.96.254:40869 was elected leader, sending a full tablet report...
I20260619 02:33:00.023836 1734 ts_tablet_manager.cc:1428] T 2fa3c1e985db4fe68da721583d4eed69 P 936121d434914b27a4cf51ad4157e40f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260619 02:33:00.024114 1735 ts_tablet_manager.cc:1428] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 02:33:00.031980 1738 raft_consensus.cc:482] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:33:00.032214 1738 raft_consensus.cc:504] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } } peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } }
I20260619 02:33:00.033399 1738 leader_election.cc:288] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 936121d434914b27a4cf51ad4157e40f (127.17.96.195:38251), 7b8fd1a8b0894b4db316d191e61e0783 (127.17.96.193:35917)
W20260619 02:33:00.036648 1452 tablet.cc:2369] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:33:00.037740 1591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:00.033681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:38251 (local address 127.17.96.194:48867)
0619 02:33:00.033831 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:00.033849 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:00.033955 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:00.034273 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:00.034277 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:00.034320 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:00.034467 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:00.034472 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:00.035489 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:00.035492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:00.036364 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:00.036374 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:00.037251 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:00.037255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:00.037380 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:00.037400 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 02:33:00.037496 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 02:33:00.037580 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"spinlock_wait_cycles":8960}
I20260619 02:33:00.037727 1730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:00.033829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:48867 (local address 127.17.96.195:38251)
0619 02:33:00.033951 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:00.033956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:00.033971 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:33:00.034105 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:00.034108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:00.034162 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:00.034231 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:00.034578 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:00.035350 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:00.036502 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:00.037136 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:00.037257 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:00.037312 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:00.037512 (+ 200us) server_negotiation.cc:299] Negotiation successful
0619 02:33:00.037591 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 02:33:00.037977 1459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:00.033798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:37829 (local address 127.17.96.193:35917)
0619 02:33:00.034012 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:00.034017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:00.034296 (+ 279us) server_negotiation.cc:407] Connection header received
0619 02:33:00.034407 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:00.034409 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:00.034462 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:00.034575 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:00.034827 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:00.035603 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:00.036728 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:00.037390 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:00.037533 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:00.037670 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:00.037737 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 02:33:00.037850 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260619 02:33:00.038417 1740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:00.033681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:35917 (local address 127.17.96.194:37829)
0619 02:33:00.034177 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:00.034188 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:00.034321 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:00.034598 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:00.034600 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:00.034619 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:00.034716 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:00.034720 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:00.035718 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:00.035721 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:00.036576 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:00.036588 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:00.037526 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:00.037530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:00.037619 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:00.037639 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 02:33:00.037719 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 02:33:00.038283 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":133,"spinlock_wait_cycles":26752,"thread_start_us":140,"threads_started":1}
I20260619 02:33:00.038508 1675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa3c1e985db4fe68da721583d4eed69" candidate_uuid: "41a83f0752ad4174925cac3c8e82faca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "936121d434914b27a4cf51ad4157e40f" is_pre_election: true
I20260619 02:33:00.038898 1675 raft_consensus.cc:2454] T 2fa3c1e985db4fe68da721583d4eed69 P 936121d434914b27a4cf51ad4157e40f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41a83f0752ad4174925cac3c8e82faca in term 0.
I20260619 02:33:00.039029 1405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa3c1e985db4fe68da721583d4eed69" candidate_uuid: "41a83f0752ad4174925cac3c8e82faca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b8fd1a8b0894b4db316d191e61e0783" is_pre_election: true
I20260619 02:33:00.039314 1405 raft_consensus.cc:2454] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41a83f0752ad4174925cac3c8e82faca in term 0.
I20260619 02:33:00.039440 1477 leader_election.cc:302] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41a83f0752ad4174925cac3c8e82faca, 936121d434914b27a4cf51ad4157e40f; no voters:
I20260619 02:33:00.039775 1738 raft_consensus.cc:2781] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:33:00.039880 1738 raft_consensus.cc:482] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:33:00.039939 1738 raft_consensus.cc:3037] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:00.041013 1738 raft_consensus.cc:504] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } } peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } }
I20260619 02:33:00.041383 1738 leader_election.cc:288] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca [CANDIDATE]: Term 1 election: Requested vote from peers 936121d434914b27a4cf51ad4157e40f (127.17.96.195:38251), 7b8fd1a8b0894b4db316d191e61e0783 (127.17.96.193:35917)
I20260619 02:33:00.041818 1675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa3c1e985db4fe68da721583d4eed69" candidate_uuid: "41a83f0752ad4174925cac3c8e82faca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "936121d434914b27a4cf51ad4157e40f"
I20260619 02:33:00.041833 1405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fa3c1e985db4fe68da721583d4eed69" candidate_uuid: "41a83f0752ad4174925cac3c8e82faca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b8fd1a8b0894b4db316d191e61e0783"
I20260619 02:33:00.041961 1405 raft_consensus.cc:3037] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:00.041961 1675 raft_consensus.cc:3037] T 2fa3c1e985db4fe68da721583d4eed69 P 936121d434914b27a4cf51ad4157e40f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:00.043305 1675 raft_consensus.cc:2454] T 2fa3c1e985db4fe68da721583d4eed69 P 936121d434914b27a4cf51ad4157e40f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41a83f0752ad4174925cac3c8e82faca in term 1.
I20260619 02:33:00.043305 1405 raft_consensus.cc:2454] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41a83f0752ad4174925cac3c8e82faca in term 1.
I20260619 02:33:00.043617 1476 leader_election.cc:302] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41a83f0752ad4174925cac3c8e82faca, 7b8fd1a8b0894b4db316d191e61e0783; no voters:
I20260619 02:33:00.043792 1738 raft_consensus.cc:2781] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:33:00.044011 1738 raft_consensus.cc:686] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca [term 1 LEADER]: Becoming Leader. State: Replica: 41a83f0752ad4174925cac3c8e82faca, State: Running, Role: LEADER
I20260619 02:33:00.044373 1738 consensus_queue.cc:237] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } } peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } }
I20260619 02:33:00.048151 1261 catalog_manager.cc:5495] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca reported cstate change: term changed from 0 to 1, leader changed from <none> to 41a83f0752ad4174925cac3c8e82faca (127.17.96.194). New cstate: current_term: 1 leader_uuid: "41a83f0752ad4174925cac3c8e82faca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } health_report { overall_health: UNKNOWN } } }
I20260619 02:33:00.069156 1732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:00.065546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35632 (local address 127.17.96.194:41259)
0619 02:33:00.065684 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:00.065688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:00.065714 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:33:00.065845 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:00.065847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:00.065909 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:00.066002 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:33:00.066234 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:00.066773 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:00.067616 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:00.067833 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:00.067907 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:00.068238 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:33:00.068688 (+ 450us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:33:00.068791 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:00.068897 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 02:33:00.068956 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 02:33:00.075513 1405 raft_consensus.cc:1261] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Refusing update from remote peer 41a83f0752ad4174925cac3c8e82faca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:33:00.075513 1675 raft_consensus.cc:1261] T 2fa3c1e985db4fe68da721583d4eed69 P 936121d434914b27a4cf51ad4157e40f [term 1 FOLLOWER]: Refusing update from remote peer 41a83f0752ad4174925cac3c8e82faca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:33:00.076284 1741 consensus_queue.cc:1035] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:33:00.076490 1738 consensus_queue.cc:1035] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca [LEADER]: Connected to new peer: Peer: permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:33:00.082979 1744 mvcc.cc:205] Tried to move back new op lower bound from 7298401812781043712 to 7298401812665032704. Current Snapshot: MvccSnapshot[applied={T|T < 7298401812781043712}]
I20260619 02:33:00.085605 1746 mvcc.cc:205] Tried to move back new op lower bound from 7298401812781043712 to 7298401812665032704. Current Snapshot: MvccSnapshot[applied={T|T < 7298401812781043712 or (T in {7298401812781043712})}]
I20260619 02:33:00.094245 1261 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:33:00.094463 1261 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:33:00.097072 1261 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2: Sending DeleteTablet for 3 replicas of tablet 2fa3c1e985db4fe68da721583d4eed69
I20260619 02:33:00.097919 1655 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fa3c1e985db4fe68da721583d4eed69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:33:00 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:35300
I20260619 02:33:00.097975 1385 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fa3c1e985db4fe68da721583d4eed69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:33:00 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:54516
I20260619 02:33:00.098424 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 1301
I20260619 02:33:00.098521 1523 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fa3c1e985db4fe68da721583d4eed69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:33:00 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:35630
I20260619 02:33:00.099174 1754 tablet_replica.cc:330] T 2fa3c1e985db4fe68da721583d4eed69 P 936121d434914b27a4cf51ad4157e40f: stopping tablet replica
I20260619 02:33:00.099347 1755 tablet_replica.cc:330] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca: stopping tablet replica
I20260619 02:33:00.099646 1755 raft_consensus.cc:2229] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:33:00.099647 1754 raft_consensus.cc:2229] T 2fa3c1e985db4fe68da721583d4eed69 P 936121d434914b27a4cf51ad4157e40f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:33:00.100057 1755 raft_consensus.cc:2258] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:33:00.100080 1754 raft_consensus.cc:2258] T 2fa3c1e985db4fe68da721583d4eed69 P 936121d434914b27a4cf51ad4157e40f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:33:00.101820 1754 ts_tablet_manager.cc:1905] T 2fa3c1e985db4fe68da721583d4eed69 P 936121d434914b27a4cf51ad4157e40f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:33:00.101820 1755 ts_tablet_manager.cc:1905] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:33:00.104761 1754 ts_tablet_manager.cc:1918] T 2fa3c1e985db4fe68da721583d4eed69 P 936121d434914b27a4cf51ad4157e40f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:33:00.104962 1754 log.cc:1196] T 2fa3c1e985db4fe68da721583d4eed69 P 936121d434914b27a4cf51ad4157e40f: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/2fa3c1e985db4fe68da721583d4eed69
I20260619 02:33:00.105031 1755 ts_tablet_manager.cc:1918] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:33:00.105119 1755 log.cc:1196] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/2fa3c1e985db4fe68da721583d4eed69
I20260619 02:33:00.105516 1754 ts_tablet_manager.cc:1939] T 2fa3c1e985db4fe68da721583d4eed69 P 936121d434914b27a4cf51ad4157e40f: Deleting consensus metadata
I20260619 02:33:00.105516 1755 ts_tablet_manager.cc:1939] T 2fa3c1e985db4fe68da721583d4eed69 P 41a83f0752ad4174925cac3c8e82faca: Deleting consensus metadata
I20260619 02:33:00.106896 1248 catalog_manager.cc:4841] TS 936121d434914b27a4cf51ad4157e40f (127.17.96.195:38251): tablet 2fa3c1e985db4fe68da721583d4eed69 (table test-table [id=b0e5d79f24bf4a21a1f6c0dd2c89d1de]) successfully deleted
I20260619 02:33:00.106947 1249 catalog_manager.cc:4841] TS 41a83f0752ad4174925cac3c8e82faca (127.17.96.194:41259): tablet 2fa3c1e985db4fe68da721583d4eed69 (table test-table [id=b0e5d79f24bf4a21a1f6c0dd2c89d1de]) successfully deleted
W20260619 02:33:00.107616 1247 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.96.193:35917 (error 108)
I20260619 02:33:00.108255 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 1455
I20260619 02:33:00.110270 1458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:00.109887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:35917 (local address 127.0.0.1:54532)
0619 02:33:00.110065 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:00.110159 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.193:35917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260619 02:33:00.110585 1247 catalog_manager.cc:4584] TS 7b8fd1a8b0894b4db316d191e61e0783 (127.17.96.193:35917): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2fa3c1e985db4fe68da721583d4eed69: Network error: Client connection negotiation failed: client connection to 127.17.96.193:35917: connect: Connection refused (error 111)
I20260619 02:33:00.115176 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 1592
I20260619 02:33:00.121172 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 1230
I20260619 02:33:20.127945 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:40869
--webserver_interface=127.17.96.254
--webserver_port=44899
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:39763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:40869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:33:20.244366 1757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:20.244668 1757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:20.244768 1757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:20.249022 1757 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 02:33:20.249112 1757 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 02:33:20.249132 1757 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 02:33:20.249150 1757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:33:20.249215 1757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:20.249264 1757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:20.249297 1757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:33:20.249339 1757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:33:20.255015 1757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:39763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:40869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:40869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=44899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:20.256256 1757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:20.257372 1757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:20.266211 1765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:20.266211 1763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:20.266319 1757 server_base.cc:1034] running on GCE node
W20260619 02:33:20.266214 1762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:20.267066 1757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:20.268254 1757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:20.269460 1757 hybrid_clock.cc:648] HybridClock initialized: now 1781836400269433 us; error 41 us; skew 500 ppm
Jun 19 02:33:20 dist-test-slave-4qw6 krb5kdc[683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836400, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:20.278095 1757 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:33:20.279150 1757 webserver.cc:460] Webserver started at http://127.17.96.254:44899/ using document root <none> and password file <none>
I20260619 02:33:20.279716 1757 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:20.279779 1757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:20.283819 1757 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:33:20.286212 1772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:20.287545 1757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:33:20.287688 1757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "60800600ea9544229691af08477568a2"
format_stamp: "Formatted at 2026-06-19 02:32:38 on dist-test-slave-4qw6"
server_key: "525651ac22f1f7824b16f1c30cf38818"
server_key_iv: "766d3624adb6bf407434b06fab1d4221"
server_key_version: "encryptionkey@0"
I20260619 02:33:20.288277 1757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:20.314213 1757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:20.317405 1757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:20.317632 1757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:20.325425 1757 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:40869
I20260619 02:33:20.325691 1824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:40869 every 8 connection(s)
I20260619 02:33:20.326752 1757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:33:20.331326 1825 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:33:20.333201 1825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2: Bootstrap starting.
I20260619 02:33:20.335783 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 1757
I20260619 02:33:20.336297 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:35917
--local_ip_for_outbound_sockets=127.17.96.193
--tserver_master_addrs=127.17.96.254:40869
--webserver_port=33085
--webserver_interface=127.17.96.193
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:39763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:33:20.351681 1825 log.cc:824] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:20.363430 1825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:33:20.363897 1825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2: Bootstrap complete.
I20260619 02:33:20.368038 1825 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60800600ea9544229691af08477568a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40869 } }
I20260619 02:33:20.368464 1825 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60800600ea9544229691af08477568a2, State: Initialized, Role: FOLLOWER
I20260619 02:33:20.369022 1825 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60800600ea9544229691af08477568a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40869 } }
I20260619 02:33:20.369174 1825 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:33:20.369263 1825 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:33:20.369381 1825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 2 FOLLOWER]: Advancing to term 3
I20260619 02:33:20.371469 1825 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60800600ea9544229691af08477568a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40869 } }
I20260619 02:33:20.371832 1825 leader_election.cc:302] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60800600ea9544229691af08477568a2; no voters:
I20260619 02:33:20.372193 1825 leader_election.cc:288] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 02:33:20.372359 1829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 02:33:20.372632 1829 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [term 3 LEADER]: Becoming Leader. State: Replica: 60800600ea9544229691af08477568a2, State: Running, Role: LEADER
I20260619 02:33:20.373037 1829 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60800600ea9544229691af08477568a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40869 } }
I20260619 02:33:20.373426 1825 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:33:20.374174 1831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60800600ea9544229691af08477568a2. Latest consensus state: current_term: 3 leader_uuid: "60800600ea9544229691af08477568a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60800600ea9544229691af08477568a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40869 } } }
I20260619 02:33:20.374327 1831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:20.374553 1830 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "60800600ea9544229691af08477568a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60800600ea9544229691af08477568a2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40869 } } }
I20260619 02:33:20.374652 1830 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:20.378619 1844 catalog_manager.cc:1259] Loaded cluster ID: 7a0f18ae2bbc481c8f01dcb6384dee6e
I20260619 02:33:20.378716 1844 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2: loading cluster ID for follower catalog manager: success
I20260619 02:33:20.380578 1844 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 60800600ea9544229691af08477568a2: acquiring CA information for follower catalog manager: success
F20260619 02:33:20.381608 1844 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f53770dd9ad google::LogMessage::Fail() at ??:0
@ 0x7f53770e088a google::LogMessage::SendToLog() at ??:0
@ 0x7f53770dd4de google::LogMessage::Flush() at ??:0
@ 0x7f53770dea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5379da118d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f5381a42989 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7f5381a1f023 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7f5381a1f54b kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7f5381a18652 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7f5381a179e9 _ZZN4kudu6master21CatalogManagerBgTasks4InitEvENKUlvE_clEv at ??:0
@ 0x7f5381a5e541 _ZNSt17_Function_handlerIFvvEZN4kudu6master21CatalogManagerBgTasks4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f5379320512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f537a9646db start_thread at ??:0
@ 0x7f5375bdb61f clone at ??:0
W20260619 02:33:20.491711 1827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:20.492158 1827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:20.492240 1827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:20.497123 1827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:20.497267 1827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:20.497377 1827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:33:20.503293 1827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:39763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:35917
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=33085
--tserver_master_addrs=127.17.96.254:40869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:20.504536 1827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:20.505664 1827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:20.516727 1865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:20.516710 1868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:20.516710 1866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:20.517118 1827 server_base.cc:1034] running on GCE node
I20260619 02:33:20.517711 1827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:20.518626 1827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:20.519850 1827 hybrid_clock.cc:648] HybridClock initialized: now 1781836400519826 us; error 36 us; skew 500 ppm
Jun 19 02:33:20 dist-test-slave-4qw6 krb5kdc[683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836400, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:20.523538 1827 init.cc:377] Logged in from keytab as kudu/127.17.96.193@KRBTEST.COM (short username kudu)
I20260619 02:33:20.524811 1827 webserver.cc:460] Webserver started at http://127.17.96.193:33085/ using document root <none> and password file <none>
I20260619 02:33:20.525593 1827 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:20.525702 1827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:20.531239 1827 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260619 02:33:20.534152 1875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:20.535844 1827 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:33:20.536006 1827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "7b8fd1a8b0894b4db316d191e61e0783"
format_stamp: "Formatted at 2026-06-19 02:32:38 on dist-test-slave-4qw6"
server_key: "c4f426952d1fce273e9814d60c83be5b"
server_key_iv: "fc9d5e0a3e142030e363f0b64ca0ae7a"
server_key_version: "encryptionkey@0"
I20260619 02:33:20.536818 1827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:20.553210 1827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:20.557129 1827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:20.557348 1827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:20.558120 1827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:33:20.559476 1882 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 02:33:20.563841 1827 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 02:33:20.563937 1827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260619 02:33:20.564018 1827 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 02:33:20.566349 1827 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 02:33:20.566423 1827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260619 02:33:20.566697 1882 tablet_bootstrap.cc:492] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783: Bootstrap starting.
I20260619 02:33:20.577718 1827 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:35917
I20260619 02:33:20.578389 1989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:35917 every 8 connection(s)
I20260619 02:33:20.579191 1827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:33:20.584388 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 1827
I20260619 02:33:20.584904 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:41259
--local_ip_for_outbound_sockets=127.17.96.194
--tserver_master_addrs=127.17.96.254:40869
--webserver_port=35503
--webserver_interface=127.17.96.194
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:39763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:33:20.588963 1882 log.cc:824] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:20.598528 1882 tablet_bootstrap.cc:492] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:33:20.599027 1882 tablet_bootstrap.cc:492] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783: Bootstrap complete.
I20260619 02:33:20.600489 1882 ts_tablet_manager.cc:1397] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783: Time spent bootstrapping tablet: real 0.034s user 0.026s sys 0.005s
I20260619 02:33:20.604058 1882 raft_consensus.cc:348] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } } peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } }
I20260619 02:33:20.604688 1882 raft_consensus.cc:729] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b8fd1a8b0894b4db316d191e61e0783, State: Initialized, Role: FOLLOWER
I20260619 02:33:20.605253 1882 consensus_queue.cc:260] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } } peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } }
I20260619 02:33:20.606503 1882 ts_tablet_manager.cc:1428] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
W20260619 02:33:20.640015 1992 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:33:20.582259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.193:58409)
0619 02:33:20.582919 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:20.582998 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:20.584300 (+ 1302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:20.639253 (+ 54953us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":166,"threads_started":1}
W20260619 02:33:20.640540 1990 heartbeater.cc:644] Failed to heartbeat to 127.17.96.254:40869 (0 consecutive failures): Network error: Failed to ping master at 127.17.96.254:40869: Client connection negotiation failed: client connection to 127.17.96.254:40869: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 02:33:20.641345 1992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:20.641086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.193:49121)
0619 02:33:20.641218 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:20.641286 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260619 02:33:20.642073 1992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:20.641847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.193:33473)
0619 02:33:20.641949 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:20.641997 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 02:33:20.642306 1990 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:33:20.713868 1993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:20.714160 1993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:20.714210 1993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:20.718600 1993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:20.718669 1993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:20.718734 1993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:33:20.724313 1993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:39763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:41259
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=35503
--tserver_master_addrs=127.17.96.254:40869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:20.725418 1993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:20.726462 1993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:33:20.736743 1993 server_base.cc:1034] running on GCE node
W20260619 02:33:20.736671 2001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:20.736672 2000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:20.736677 2003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:20.737327 1993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:20.738106 1993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:20.739274 1993 hybrid_clock.cc:648] HybridClock initialized: now 1781836400739259 us; error 28 us; skew 500 ppm
Jun 19 02:33:20 dist-test-slave-4qw6 krb5kdc[683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836400, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:20.742118 1993 init.cc:377] Logged in from keytab as kudu/127.17.96.194@KRBTEST.COM (short username kudu)
I20260619 02:33:20.743139 1993 webserver.cc:460] Webserver started at http://127.17.96.194:35503/ using document root <none> and password file <none>
I20260619 02:33:20.743803 1993 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:20.743899 1993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:20.748260 1993 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:33:20.750699 2010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:20.752085 1993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 02:33:20.752244 1993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "41a83f0752ad4174925cac3c8e82faca"
format_stamp: "Formatted at 2026-06-19 02:32:38 on dist-test-slave-4qw6"
server_key: "0faab3067808fdde02eb390541b51e94"
server_key_iv: "cfa8b83f2d01a0aa8c80e803d91834c9"
server_key_version: "encryptionkey@0"
I20260619 02:33:20.752847 1993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:20.777604 1993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:20.780831 1993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:20.781085 1993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:20.781730 1993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:33:20.782641 1993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:33:20.782716 1993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:20.782790 1993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:33:20.782842 1993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:20.792825 1993 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:41259
I20260619 02:33:20.792845 2123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:41259 every 8 connection(s)
I20260619 02:33:20.793994 1993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:33:20.797822 2126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:20.796107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.194:42169)
0619 02:33:20.796559 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:20.796668 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":113,"threads_started":1}
W20260619 02:33:20.798355 2124 heartbeater.cc:644] Failed to heartbeat to 127.17.96.254:40869 (0 consecutive failures): Network error: Failed to ping master at 127.17.96.254:40869: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
I20260619 02:33:20.798939 2126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:20.798724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.194:59739)
0619 02:33:20.798835 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:20.798881 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260619 02:33:20.799520 2126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:20.799340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.194:38523)
0619 02:33:20.799434 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:20.799474 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260619 02:33:20.799710 2124 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 02:33:20.799726 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 1993
I20260619 02:33:20.800136 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:38251
--local_ip_for_outbound_sockets=127.17.96.195
--tserver_master_addrs=127.17.96.254:40869
--webserver_port=46701
--webserver_interface=127.17.96.195
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:39763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:33:20.913552 2127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:20.913926 2127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:20.914014 2127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:20.918296 2127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:20.918395 2127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:20.918497 2127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:33:20.924078 2127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:39763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:38251
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=46701
--tserver_master_addrs=127.17.96.254:40869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:20.925257 2127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:20.926338 2127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:20.936643 2135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:20.936658 2132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:20.936662 2133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:20.936995 2127 server_base.cc:1034] running on GCE node
I20260619 02:33:20.937460 2127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:20.938212 2127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:20.939404 2127 hybrid_clock.cc:648] HybridClock initialized: now 1781836400939364 us; error 54 us; skew 500 ppm
Jun 19 02:33:20 dist-test-slave-4qw6 krb5kdc[683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836400, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:20.942278 2127 init.cc:377] Logged in from keytab as kudu/127.17.96.195@KRBTEST.COM (short username kudu)
I20260619 02:33:20.943279 2127 webserver.cc:460] Webserver started at http://127.17.96.195:46701/ using document root <none> and password file <none>
I20260619 02:33:20.943888 2127 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:20.943984 2127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:20.948048 2127 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:33:20.950202 2142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:20.951421 2127 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 02:33:20.951560 2127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "936121d434914b27a4cf51ad4157e40f"
format_stamp: "Formatted at 2026-06-19 02:32:38 on dist-test-slave-4qw6"
server_key: "f1ff164a8daf50f9188d293e435c79fb"
server_key_iv: "ad8b7532e6bccfc955570ff9f97b9299"
server_key_version: "encryptionkey@0"
I20260619 02:33:20.952143 2127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:20.975072 2127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:20.978307 2127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:20.978538 2127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:20.979182 2127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:33:20.980080 2127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:33:20.980155 2127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:20.980229 2127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:33:20.980262 2127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:20.989578 2127 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:38251
I20260619 02:33:20.989604 2255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:38251 every 8 connection(s)
I20260619 02:33:20.990643 2127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:33:20.994614 2258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:20.992822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.195:41245)
0619 02:33:20.993297 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:20.993412 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":115,"threads_started":1}
W20260619 02:33:20.995100 2256 heartbeater.cc:644] Failed to heartbeat to 127.17.96.254:40869 (0 consecutive failures): Network error: Failed to ping master at 127.17.96.254:40869: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
I20260619 02:33:20.995738 2258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:20.995521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.195:57425)
0619 02:33:20.995630 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:20.995681 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 02:33:20.996376 2258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:20.996191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.195:32971)
0619 02:33:20.996285 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:20.996326 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260619 02:33:20.996591 2256 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 02:33:20.997262 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 2127
I20260619 02:33:21.643344 2260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:21.642923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.193:45391)
0619 02:33:21.643169 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:21.643228 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":74,"threads_started":1}
I20260619 02:33:21.800762 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:21.800328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.194:49041)
0619 02:33:21.800585 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:21.800649 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":72,"threads_started":1}
I20260619 02:33:21.925253 2262 raft_consensus.cc:482] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:33:21.925449 2262 raft_consensus.cc:504] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } } peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } }
I20260619 02:33:21.926580 2262 leader_election.cc:288] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 936121d434914b27a4cf51ad4157e40f (127.17.96.195:38251), 41a83f0752ad4174925cac3c8e82faca (127.17.96.194:41259)
Jun 19 02:33:21 dist-test-slave-4qw6 krb5kdc[683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836400, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.194@KRBTEST.COM
Jun 19 02:33:21 dist-test-slave-4qw6 krb5kdc[683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836400, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.195@KRBTEST.COM
I20260619 02:33:21.941538 2264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:21.927041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:33895 (local address 127.17.96.195:38251)
0619 02:33:21.927473 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:21.927493 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:21.927542 (+ 49us) server_negotiation.cc:407] Connection header received
0619 02:33:21.928438 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:21.928447 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:21.928957 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:21.929312 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:21.930910 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:21.931895 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:21.933079 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:21.933549 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:21.933732 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:21.936880 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:21.936885 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:21.936896 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:33:21.936925 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:21.938722 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:21.939302 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:21.939314 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:21.939324 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:21.939442 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:21.939826 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:21.939829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:21.939831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:21.940334 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:21.940467 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:21.941065 (+ 598us) server_negotiation.cc:299] Negotiation successful
0619 02:33:21.941339 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":92,"threads_started":1}
I20260619 02:33:21.941522 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:21.926847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:38251 (local address 127.17.96.193:33895)
0619 02:33:21.927225 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:21.927242 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:21.927355 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:21.929835 (+ 2480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:21.929838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:21.930160 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:33:21.930701 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:21.930727 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:21.932057 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:21.932059 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:21.932949 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:21.932958 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:21.933679 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:21.933683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:21.934053 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:21.934681 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:33:21.934693 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:33:21.936680 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:33:21.938874 (+ 2194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:33:21.938879 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:33:21.938881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:33:21.939168 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:33:21.939543 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:33:21.939548 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:33:21.939550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:33:21.939715 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:33:21.940518 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:33:21.940521 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:33:21.940723 (+ 202us) client_negotiation.cc:769] Sending connection context
0619 02:33:21.940949 (+ 226us) client_negotiation.cc:240] Negotiation successful
0619 02:33:21.941348 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":90,"spinlock_wait_cycles":8704,"thread_start_us":67,"threads_started":1}
I20260619 02:33:21.941677 2260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:21.926847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:41259 (local address 127.17.96.193:54871)
0619 02:33:21.927066 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:21.927083 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:21.927204 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:21.929538 (+ 2334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:21.929545 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:21.930132 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:33:21.930701 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:21.930720 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:21.932025 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:21.932031 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:21.932952 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:21.932961 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:21.933689 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:21.933694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:21.933889 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:21.934640 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:33:21.934657 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:33:21.936350 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:33:21.938794 (+ 2444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:33:21.938800 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:33:21.938815 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:33:21.939155 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:33:21.939533 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:33:21.939537 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:33:21.939539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:33:21.939697 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:33:21.940445 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:33:21.940452 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:33:21.940726 (+ 274us) client_negotiation.cc:769] Sending connection context
0619 02:33:21.940948 (+ 222us) client_negotiation.cc:240] Negotiation successful
0619 02:33:21.941464 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":347,"spinlock_wait_cycles":13312}
I20260619 02:33:21.941690 2265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:21.927099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:54871 (local address 127.17.96.194:41259)
0619 02:33:21.927486 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:21.927506 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:21.927541 (+ 35us) server_negotiation.cc:407] Connection header received
0619 02:33:21.928452 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:21.928458 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:21.928963 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:21.929312 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:21.930897 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:21.931872 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:21.933088 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:21.933551 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:21.933724 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:21.936591 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:21.936596 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:21.936610 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:33:21.936641 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:21.938635 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:21.939292 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:21.939302 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:21.939307 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:21.939433 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:21.939797 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:21.939801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:21.939802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:21.940314 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:21.940465 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:21.941211 (+ 746us) server_negotiation.cc:299] Negotiation successful
0619 02:33:21.941504 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":115,"threads_started":1}
W20260619 02:33:21.944366 1877 leader_election.cc:341] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 936121d434914b27a4cf51ad4157e40f (127.17.96.195:38251): Not found: Tablet not found: 2fa3c1e985db4fe68da721583d4eed69
W20260619 02:33:21.944581 1878 leader_election.cc:341] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 41a83f0752ad4174925cac3c8e82faca (127.17.96.194:41259): Not found: Tablet not found: 2fa3c1e985db4fe68da721583d4eed69
I20260619 02:33:21.944656 1878 leader_election.cc:302] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b8fd1a8b0894b4db316d191e61e0783; no voters: 41a83f0752ad4174925cac3c8e82faca, 936121d434914b27a4cf51ad4157e40f
I20260619 02:33:21.944906 2262 raft_consensus.cc:2726] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:33:21.997452 2266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:21.997078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.195:51311)
0619 02:33:21.997305 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:21.997359 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":68,"threads_started":1}
I20260619 02:33:22.644599 2267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:22.644155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.193:38529)
0619 02:33:22.644431 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:22.644497 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":82,"threads_started":1}
I20260619 02:33:22.802037 2268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:22.801605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.194:46211)
0619 02:33:22.801891 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:22.801942 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":71,"threads_started":1}
I20260619 02:33:22.998854 2269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:22.998417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.195:45695)
0619 02:33:22.998678 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:22.998737 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":73,"threads_started":1}
I20260619 02:33:23.644706 2270 raft_consensus.cc:482] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:33:23.644879 2270 raft_consensus.cc:504] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } } peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } }
I20260619 02:33:23.645288 2270 leader_election.cc:288] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 936121d434914b27a4cf51ad4157e40f (127.17.96.195:38251), 41a83f0752ad4174925cac3c8e82faca (127.17.96.194:41259)
I20260619 02:33:23.645854 2271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:23.645406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.193:33635)
0619 02:33:23.645628 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:23.645724 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":64,"threads_started":1}
W20260619 02:33:23.646240 1878 leader_election.cc:341] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 41a83f0752ad4174925cac3c8e82faca (127.17.96.194:41259): Not found: Tablet not found: 2fa3c1e985db4fe68da721583d4eed69
W20260619 02:33:23.646348 1877 leader_election.cc:341] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 936121d434914b27a4cf51ad4157e40f (127.17.96.195:38251): Not found: Tablet not found: 2fa3c1e985db4fe68da721583d4eed69
I20260619 02:33:23.646525 1877 leader_election.cc:302] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b8fd1a8b0894b4db316d191e61e0783; no voters: 41a83f0752ad4174925cac3c8e82faca, 936121d434914b27a4cf51ad4157e40f
I20260619 02:33:23.646689 2270 raft_consensus.cc:2726] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:33:23.803303 2272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:23.802856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.194:52693)
0619 02:33:23.803133 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:23.803200 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":95,"threads_started":1}
I20260619 02:33:24.000192 2273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:23.999746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.195:44215)
0619 02:33:24.000034 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:24.000097 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":89,"threads_started":1}
I20260619 02:33:24.647368 2274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:24.646863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.193:45205)
0619 02:33:24.647193 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:24.647259 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":110,"threads_started":1}
I20260619 02:33:24.804524 2275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:24.804103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.194:49923)
0619 02:33:24.804362 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:24.804426 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":76,"threads_started":1}
I20260619 02:33:25.001482 2276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:25.001052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.195:58751)
0619 02:33:25.001327 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:25.001384 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":109,"threads_started":1}
I20260619 02:33:25.648819 2277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:25.648352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.193:34081)
0619 02:33:25.648656 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:25.648717 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":95,"threads_started":1}
I20260619 02:33:25.806102 2278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:25.805559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.194:57031)
0619 02:33:25.805923 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:25.805995 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":88,"threads_started":1}
I20260619 02:33:25.925208 2279 raft_consensus.cc:482] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:33:25.925369 2279 raft_consensus.cc:504] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } } peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } }
I20260619 02:33:25.925900 2279 leader_election.cc:288] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 936121d434914b27a4cf51ad4157e40f (127.17.96.195:38251), 41a83f0752ad4174925cac3c8e82faca (127.17.96.194:41259)
W20260619 02:33:25.926728 1877 leader_election.cc:341] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 936121d434914b27a4cf51ad4157e40f (127.17.96.195:38251): Not found: Tablet not found: 2fa3c1e985db4fe68da721583d4eed69
W20260619 02:33:25.926826 1878 leader_election.cc:341] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 41a83f0752ad4174925cac3c8e82faca (127.17.96.194:41259): Not found: Tablet not found: 2fa3c1e985db4fe68da721583d4eed69
I20260619 02:33:25.926895 1878 leader_election.cc:302] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b8fd1a8b0894b4db316d191e61e0783; no voters: 41a83f0752ad4174925cac3c8e82faca, 936121d434914b27a4cf51ad4157e40f
I20260619 02:33:25.927042 2279 raft_consensus.cc:2726] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:33:26.002915 2280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:26.002436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.195:50867)
0619 02:33:26.002744 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:26.002812 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":81,"threads_started":1}
I20260619 02:33:26.650015 2281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:26.649569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.193:58025)
0619 02:33:26.649867 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:26.649918 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":86,"threads_started":1}
I20260619 02:33:26.807348 2282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:26.806920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.194:40415)
0619 02:33:26.807191 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:26.807251 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":90,"threads_started":1}
I20260619 02:33:27.004185 2283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:27.003761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.195:48905)
0619 02:33:27.004026 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:27.004087 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":83,"threads_started":1}
I20260619 02:33:27.651310 2284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:27.650844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.193:59839)
0619 02:33:27.651138 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:27.651207 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":85,"threads_started":1}
I20260619 02:33:27.808868 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:27.808289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.194:44329)
0619 02:33:27.808652 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:27.808741 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":94,"threads_started":1}
I20260619 02:33:28.005705 2286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:28.005198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.195:55459)
0619 02:33:28.005539 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:28.005605 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":98,"threads_started":1}
I20260619 02:33:28.652551 2287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:28.652134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.193:59559)
0619 02:33:28.652382 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:28.652442 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":79,"threads_started":1}
I20260619 02:33:28.810187 2288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:28.809734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.194:57767)
0619 02:33:28.810027 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:28.810090 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":74,"threads_started":1}
I20260619 02:33:29.007019 2289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:29.006586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.195:50399)
0619 02:33:29.006856 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:29.006921 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":90,"threads_started":1}
I20260619 02:33:29.654094 2290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:29.653511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.193:47931)
0619 02:33:29.653865 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:29.653956 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":82,"threads_started":1}
I20260619 02:33:29.811480 2291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:29.811059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.194:45263)
0619 02:33:29.811328 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:29.811382 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":93,"threads_started":1}
I20260619 02:33:30.008567 2292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:30.008033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.195:48873)
0619 02:33:30.008349 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:30.008416 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":97,"threads_started":1}
I20260619 02:33:30.655539 2293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:30.655091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.193:42615)
0619 02:33:30.655372 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:30.655432 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":106,"threads_started":1}
I20260619 02:33:30.812884 2294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:30.812324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.194:34839)
0619 02:33:30.812718 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:30.812786 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":104,"threads_started":1}
I20260619 02:33:30.966764 2295 raft_consensus.cc:482] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:33:30.966902 2295 raft_consensus.cc:504] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } } peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } }
I20260619 02:33:30.967356 2295 leader_election.cc:288] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 936121d434914b27a4cf51ad4157e40f (127.17.96.195:38251), 41a83f0752ad4174925cac3c8e82faca (127.17.96.194:41259)
W20260619 02:33:30.968115 1877 leader_election.cc:341] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 936121d434914b27a4cf51ad4157e40f (127.17.96.195:38251): Not found: Tablet not found: 2fa3c1e985db4fe68da721583d4eed69
W20260619 02:33:30.968200 1878 leader_election.cc:341] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 41a83f0752ad4174925cac3c8e82faca (127.17.96.194:41259): Not found: Tablet not found: 2fa3c1e985db4fe68da721583d4eed69
I20260619 02:33:30.968325 1878 leader_election.cc:302] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b8fd1a8b0894b4db316d191e61e0783; no voters: 41a83f0752ad4174925cac3c8e82faca, 936121d434914b27a4cf51ad4157e40f
I20260619 02:33:30.968511 2295 raft_consensus.cc:2726] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:33:31.009996 2296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:31.009481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.195:33073)
0619 02:33:31.009826 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:31.009893 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":104,"threads_started":1}
I20260619 02:33:31.656975 2297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:31.656421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.193:46857)
0619 02:33:31.656723 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:31.656831 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":81,"threads_started":1}
I20260619 02:33:31.814198 2298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:31.813697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.194:49681)
0619 02:33:31.814045 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:31.814102 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":173,"threads_started":1}
I20260619 02:33:32.011370 2299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:32.010884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.195:41839)
0619 02:33:32.011177 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:32.011233 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":104,"threads_started":1}
I20260619 02:33:32.658314 2300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:32.657888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.193:43701)
0619 02:33:32.658155 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:32.658217 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":82,"threads_started":1}
I20260619 02:33:32.815521 2301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:32.815082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.194:46835)
0619 02:33:32.815355 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:32.815421 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":82,"threads_started":1}
I20260619 02:33:33.012770 2302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:33.012296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.195:55771)
0619 02:33:33.012571 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:33.012641 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":81,"threads_started":1}
I20260619 02:33:33.659493 2303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:33.659087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.193:60435)
0619 02:33:33.659328 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:33.659386 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":74,"threads_started":1}
I20260619 02:33:33.816891 2304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:33.816367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.194:40607)
0619 02:33:33.816724 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:33.816792 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":97,"threads_started":1}
I20260619 02:33:34.014192 2305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:34.013691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.195:34195)
0619 02:33:34.014027 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:34.014089 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":138,"threads_started":1}
I20260619 02:33:34.082301 2306 raft_consensus.cc:482] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:33:34.082466 2306 raft_consensus.cc:504] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "936121d434914b27a4cf51ad4157e40f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38251 } } peers { permanent_uuid: "41a83f0752ad4174925cac3c8e82faca" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 41259 } } peers { permanent_uuid: "7b8fd1a8b0894b4db316d191e61e0783" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35917 } }
I20260619 02:33:34.082942 2306 leader_election.cc:288] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 936121d434914b27a4cf51ad4157e40f (127.17.96.195:38251), 41a83f0752ad4174925cac3c8e82faca (127.17.96.194:41259)
W20260619 02:33:34.083773 1877 leader_election.cc:341] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 936121d434914b27a4cf51ad4157e40f (127.17.96.195:38251): Not found: Tablet not found: 2fa3c1e985db4fe68da721583d4eed69
W20260619 02:33:34.083921 1878 leader_election.cc:341] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 41a83f0752ad4174925cac3c8e82faca (127.17.96.194:41259): Not found: Tablet not found: 2fa3c1e985db4fe68da721583d4eed69
I20260619 02:33:34.084002 1878 leader_election.cc:302] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b8fd1a8b0894b4db316d191e61e0783; no voters: 41a83f0752ad4174925cac3c8e82faca, 936121d434914b27a4cf51ad4157e40f
I20260619 02:33:34.084184 2306 raft_consensus.cc:2726] T 2fa3c1e985db4fe68da721583d4eed69 P 7b8fd1a8b0894b4db316d191e61e0783 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:33:34.660923 2307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:34.660407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.193:55327)
0619 02:33:34.660754 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:34.660821 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":79,"threads_started":1}
I20260619 02:33:34.818356 2308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:34.817888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.194:37505)
0619 02:33:34.818164 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:34.818239 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":102,"threads_started":1}
I20260619 02:33:35.015733 2309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:35.015116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.195:56131)
0619 02:33:35.015554 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:35.015628 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":104,"threads_started":1}
I20260619 02:33:35.662485 2310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:35.662046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.193:58241)
0619 02:33:35.662326 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:35.662386 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":85,"threads_started":1}
I20260619 02:33:35.819818 2311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:35.819379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:40869 (local address 127.17.96.194:42397)
0619 02:33:35.819652 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:35.819714 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.254:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":95,"threads_started":1}
I20260619 02:33:36.002609 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 1827
I20260619 02:33:36.008715 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 1993
I20260619 02:33:36.014300 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 2127
2026-06-19T02:33:36Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59276 ms)
[----------] 30 tests from SecurityITest (254001 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:33:36 dist-test-slave-4qw6 krb5kdc[2314](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:33:36 dist-test-slave-4qw6 krb5kdc[2314](info): set up 2 sockets
Jun 19 02:33:36 dist-test-slave-4qw6 krb5kdc[2314](info): commencing operation
krb5kdc: starting...
W20260619 02:33:37.065939 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:33:37 dist-test-slave-4qw6 krb5kdc[2314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:33:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:33:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:33:37.192795 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:45455
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:41971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:45455
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:33:37.310388 2327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:37.310683 2327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:37.310726 2327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:37.314872 2327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:33:37.314944 2327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:37.314965 2327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:37.314982 2327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:33:37.314998 2327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:33:37.320595 2327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:41971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:45455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:45455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:37.321815 2327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:37.322887 2327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:33:37.332185 2327 server_base.cc:1034] running on GCE node
W20260619 02:33:37.332129 2332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:37.332146 2333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:37.332129 2335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:37.333014 2327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:37.334271 2327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:37.335438 2327 hybrid_clock.cc:648] HybridClock initialized: now 1781836417335418 us; error 45 us; skew 500 ppm
Jun 19 02:33:37 dist-test-slave-4qw6 krb5kdc[2314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836417, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:37.341509 2327 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:33:37.342604 2327 webserver.cc:460] Webserver started at http://127.17.96.254:44509/ using document root <none> and password file <none>
I20260619 02:33:37.343210 2327 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:37.343281 2327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:37.343532 2327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:37.345804 2327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "d4252aca2b354406999e3ab7c6a30968"
format_stamp: "Formatted at 2026-06-19 02:33:37 on dist-test-slave-4qw6"
server_key: "ee61d6c931e9a11b4962f7ca525cced8"
server_key_iv: "1d3f8d545feb94bdd486c22f2d3724c8"
server_key_version: "encryptionkey@0"
I20260619 02:33:37.346446 2327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "d4252aca2b354406999e3ab7c6a30968"
format_stamp: "Formatted at 2026-06-19 02:33:37 on dist-test-slave-4qw6"
server_key: "ee61d6c931e9a11b4962f7ca525cced8"
server_key_iv: "1d3f8d545feb94bdd486c22f2d3724c8"
server_key_version: "encryptionkey@0"
I20260619 02:33:37.350508 2327 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 02:33:37.353163 2342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:37.354529 2327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:33:37.354679 2327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "d4252aca2b354406999e3ab7c6a30968"
format_stamp: "Formatted at 2026-06-19 02:33:37 on dist-test-slave-4qw6"
server_key: "ee61d6c931e9a11b4962f7ca525cced8"
server_key_iv: "1d3f8d545feb94bdd486c22f2d3724c8"
server_key_version: "encryptionkey@0"
I20260619 02:33:37.354809 2327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:37.408411 2327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:37.411617 2327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:37.411851 2327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:37.419239 2327 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:45455
I20260619 02:33:37.419257 2394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:45455 every 8 connection(s)
I20260619 02:33:37.420320 2327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:33:37.423171 2395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:37.429077 2395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968: Bootstrap starting.
I20260619 02:33:37.430085 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 2327
I20260619 02:33:37.430241 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:33:37.430541 17795 external_mini_cluster.cc:1428] Setting key c44bfce31bc38b316348dde07876e4f2
I20260619 02:33:37.432215 2395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:37.433140 2395 log.cc:824] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:37.435582 2395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968: No bootstrap required, opened a new log
I20260619 02:33:37.440027 2395 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4252aca2b354406999e3ab7c6a30968" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45455 } }
I20260619 02:33:37.440315 2395 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Jun 19 02:33:37 dist-test-slave-4qw6 krb5kdc[2314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:33:37.440409 2395 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4252aca2b354406999e3ab7c6a30968, State: Initialized, Role: FOLLOWER
I20260619 02:33:37.440984 2395 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4252aca2b354406999e3ab7c6a30968" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45455 } }
I20260619 02:33:37.441145 2395 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:33:37.441258 2395 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:33:37.441360 2395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:37.442425 2395 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4252aca2b354406999e3ab7c6a30968" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45455 } }
I20260619 02:33:37.442809 2395 leader_election.cc:302] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4252aca2b354406999e3ab7c6a30968; no voters:
I20260619 02:33:37.443164 2395 leader_election.cc:288] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:33:37.443277 2400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:33:37.443514 2400 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968 [term 1 LEADER]: Becoming Leader. State: Replica: d4252aca2b354406999e3ab7c6a30968, State: Running, Role: LEADER
I20260619 02:33:37.443925 2400 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4252aca2b354406999e3ab7c6a30968" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45455 } }
I20260619 02:33:37.444573 2395 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:33:37.445708 2401 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4252aca2b354406999e3ab7c6a30968" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4252aca2b354406999e3ab7c6a30968" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45455 } } }
I20260619 02:33:37.445855 2401 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968 [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:37.446266 2402 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4252aca2b354406999e3ab7c6a30968. Latest consensus state: current_term: 1 leader_uuid: "d4252aca2b354406999e3ab7c6a30968" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4252aca2b354406999e3ab7c6a30968" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45455 } } }
I20260619 02:33:37.446415 2402 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968 [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:37.446705 2409 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:33:37.447324 2398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:37.431762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51788 (local address 127.17.96.254:45455)
0619 02:33:37.432288 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:37.432308 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:37.432351 (+ 43us) server_negotiation.cc:407] Connection header received
0619 02:33:37.433305 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:37.433330 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:37.433832 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:37.434224 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:37.435024 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:37.436517 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:37.437439 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:37.438207 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:37.438401 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:37.441039 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:37.441045 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:37.441059 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:33:37.441092 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:37.443207 (+ 2115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:37.443812 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:37.443818 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:37.443824 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:37.443982 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:37.444296 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:37.444299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:37.444300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:37.444806 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:37.445024 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:37.445867 (+ 843us) server_negotiation.cc:299] Negotiation successful
0619 02:33:37.446216 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":132,"threads_started":1}
I20260619 02:33:37.449606 2409 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:33:37.455482 2409 catalog_manager.cc:1347] Generated new cluster ID: a62b7503c66a41f691e74d2a4adef932
I20260619 02:33:37.455576 2409 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:33:37.477145 2409 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:33:37.478734 2409 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:33:37.488773 2409 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968: Generated new TSK 0
I20260619 02:33:37.489578 2409 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.96.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
I20260619 02:33:37.524462 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45455
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:41971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:33:37.641805 2421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:37.642100 2421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:37.642174 2421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:37.646682 2421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:37.646780 2421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:37.646914 2421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:33:37.652644 2421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:41971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:37.653839 2421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:37.654876 2421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:37.664697 2426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:37.664794 2421 server_base.cc:1034] running on GCE node
W20260619 02:33:37.664698 2429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:37.664729 2427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:37.665380 2421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:37.666165 2421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:37.667330 2421 hybrid_clock.cc:648] HybridClock initialized: now 1781836417667297 us; error 46 us; skew 500 ppm
Jun 19 02:33:37 dist-test-slave-4qw6 krb5kdc[2314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836417, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:37.670315 2421 init.cc:377] Logged in from keytab as kudu/127.17.96.193@KRBTEST.COM (short username kudu)
I20260619 02:33:37.671368 2421 webserver.cc:460] Webserver started at http://127.17.96.193:43991/ using document root <none> and password file <none>
I20260619 02:33:37.671991 2421 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:37.672091 2421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:37.672330 2421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:37.674610 2421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "826d070a24b149729b184ac7a161b0b9"
format_stamp: "Formatted at 2026-06-19 02:33:37 on dist-test-slave-4qw6"
server_key: "dfb441bd4e4893e3cecd2389c5bd1d94"
server_key_iv: "bd983d9ecb833a5ac152d85a721cf367"
server_key_version: "encryptionkey@0"
I20260619 02:33:37.675227 2421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "826d070a24b149729b184ac7a161b0b9"
format_stamp: "Formatted at 2026-06-19 02:33:37 on dist-test-slave-4qw6"
server_key: "dfb441bd4e4893e3cecd2389c5bd1d94"
server_key_iv: "bd983d9ecb833a5ac152d85a721cf367"
server_key_version: "encryptionkey@0"
I20260619 02:33:37.679239 2421 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 02:33:37.682163 2436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:37.683761 2421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 02:33:37.683915 2421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "826d070a24b149729b184ac7a161b0b9"
format_stamp: "Formatted at 2026-06-19 02:33:37 on dist-test-slave-4qw6"
server_key: "dfb441bd4e4893e3cecd2389c5bd1d94"
server_key_iv: "bd983d9ecb833a5ac152d85a721cf367"
server_key_version: "encryptionkey@0"
I20260619 02:33:37.684051 2421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:37.700510 2421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:37.703895 2421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:37.704126 2421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:37.704869 2421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:33:37.705912 2421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:33:37.705962 2421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:37.706007 2421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:33:37.706028 2421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:37.715837 2421 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:39739
I20260619 02:33:37.715859 2549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:39739 every 8 connection(s)
I20260619 02:33:37.716810 2421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:33:37.721642 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 2421
I20260619 02:33:37.721835 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:33:37.722131 17795 external_mini_cluster.cc:1428] Setting key f59e6b976462b9c9e4e709a3ef9737be
Jun 19 02:33:37 dist-test-slave-4qw6 krb5kdc[2314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836417, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.96.194@KRBTEST.COM; defaulting to no policy
I20260619 02:33:37.732177 2398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:37.718736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:59953 (local address 127.17.96.254:45455)
0619 02:33:37.718889 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:37.718893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:37.719747 (+ 854us) server_negotiation.cc:407] Connection header received
0619 02:33:37.721168 (+ 1421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:37.721172 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:37.721236 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:37.721317 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:37.722751 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:37.723317 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:37.724611 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:37.724874 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:37.725022 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:37.727907 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:37.727911 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:37.727913 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:33:37.727942 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:37.729606 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:37.730287 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:37.730292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:37.730296 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:37.730404 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:37.730822 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:37.730829 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:37.730830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:37.731068 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:37.731191 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:37.731799 (+ 608us) server_negotiation.cc:299] Negotiation successful
0619 02:33:37.731966 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 02:33:37.733253 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:37.719117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45455 (local address 127.17.96.193:59953)
0619 02:33:37.719590 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:37.719636 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:37.720918 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:37.721502 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:37.721509 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:37.722012 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:33:37.722583 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:37.722599 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:37.723482 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:37.723485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:37.724435 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:37.724446 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:37.725076 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:37.725080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:37.725279 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:37.726027 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:33:37.726046 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:33:37.727674 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:33:37.729813 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:33:37.729821 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:33:37.729842 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:33:37.730158 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:33:37.730516 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:33:37.730520 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:33:37.730522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:33:37.730700 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:33:37.731192 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:33:37.731202 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:33:37.731521 (+ 319us) client_negotiation.cc:769] Sending connection context
0619 02:33:37.731738 (+ 217us) client_negotiation.cc:240] Negotiation successful
0619 02:33:37.732153 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":125,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.194@KRBTEST.COM" created.
I20260619 02:33:37.734887 2550 heartbeater.cc:347] Connected to a master server at 127.17.96.254:45455
I20260619 02:33:37.735209 2550 heartbeater.cc:464] Registering TS with master...
I20260619 02:33:37.735857 2550 heartbeater.cc:511] Master 127.17.96.254:45455 requested a full tablet report, sending...
I20260619 02:33:37.737951 2359 ts_manager.cc:194] Registered new tserver with Master: 826d070a24b149729b184ac7a161b0b9 (127.17.96.193:39739)
I20260619 02:33:37.739779 2359 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:59953
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
I20260619 02:33:37.751124 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45455
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:41971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:33:37.865327 2555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:37.865633 2555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:37.865679 2555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:37.870133 2555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:37.870213 2555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:37.870321 2555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:33:37.876139 2555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:41971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:37.877393 2555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:37.878564 2555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:37.888814 2563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:37.888983 2555 server_base.cc:1034] running on GCE node
W20260619 02:33:37.888818 2561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:37.889171 2560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:37.889616 2555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:37.890353 2555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:37.891546 2555 hybrid_clock.cc:648] HybridClock initialized: now 1781836417891514 us; error 47 us; skew 500 ppm
Jun 19 02:33:37 dist-test-slave-4qw6 krb5kdc[2314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836417, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:37.894402 2555 init.cc:377] Logged in from keytab as kudu/127.17.96.194@KRBTEST.COM (short username kudu)
I20260619 02:33:37.895291 2555 webserver.cc:460] Webserver started at http://127.17.96.194:46393/ using document root <none> and password file <none>
I20260619 02:33:37.895849 2555 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:37.895911 2555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:37.896082 2555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:37.898303 2555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "d70a1ba6a94e4c1eb4e130590ce5f106"
format_stamp: "Formatted at 2026-06-19 02:33:37 on dist-test-slave-4qw6"
server_key: "c3880aa250e359fca9898d926481aef4"
server_key_iv: "37aed85486e74e3b0ffa65216a148f11"
server_key_version: "encryptionkey@0"
I20260619 02:33:37.898864 2555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "d70a1ba6a94e4c1eb4e130590ce5f106"
format_stamp: "Formatted at 2026-06-19 02:33:37 on dist-test-slave-4qw6"
server_key: "c3880aa250e359fca9898d926481aef4"
server_key_iv: "37aed85486e74e3b0ffa65216a148f11"
server_key_version: "encryptionkey@0"
I20260619 02:33:37.902763 2555 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 02:33:37.905185 2570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:37.906538 2555 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 02:33:37.906682 2555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "d70a1ba6a94e4c1eb4e130590ce5f106"
format_stamp: "Formatted at 2026-06-19 02:33:37 on dist-test-slave-4qw6"
server_key: "c3880aa250e359fca9898d926481aef4"
server_key_iv: "37aed85486e74e3b0ffa65216a148f11"
server_key_version: "encryptionkey@0"
I20260619 02:33:37.906805 2555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:37.917145 2555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:37.920207 2555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:37.920442 2555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:37.921090 2555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:33:37.922068 2555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:33:37.922119 2555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:37.922192 2555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:33:37.922235 2555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:37.931319 2555 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:43947
I20260619 02:33:37.931391 2683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:43947 every 8 connection(s)
I20260619 02:33:37.932294 2555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:33:37.938710 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 2555
I20260619 02:33:37.938850 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:33:37.939143 17795 external_mini_cluster.cc:1428] Setting key e9a220887ac973d683a3a7b84eab84de
Jun 19 02:33:37 dist-test-slave-4qw6 krb5kdc[2314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836417, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:33:37.947755 2398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:37.934271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:47715 (local address 127.17.96.254:45455)
0619 02:33:37.934411 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:37.934414 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:37.935274 (+ 860us) server_negotiation.cc:407] Connection header received
0619 02:33:37.936668 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:37.936672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:37.936735 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:37.936839 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:37.938117 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:37.938795 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:37.940202 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:37.940423 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:37.940564 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:37.943275 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:37.943280 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:37.943285 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:33:37.943319 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:37.945041 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:37.945863 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:37.945868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:37.945871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:37.945998 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:37.946490 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:37.946492 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:37.946493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:37.946685 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:37.946760 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:37.947509 (+ 749us) server_negotiation.cc:299] Negotiation successful
0619 02:33:37.947627 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 02:33:37.949061 2686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:37.934649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45455 (local address 127.17.96.194:47715)
0619 02:33:37.935116 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:37.935163 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:37.936437 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:37.937019 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:37.937025 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:37.937385 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:33:37.937962 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:37.937978 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:37.938969 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:37.938973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:37.940070 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:37.940081 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:37.940611 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:37.940615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:37.940794 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:37.941422 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:33:37.941438 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:33:37.943085 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:33:37.945196 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:33:37.945207 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:33:37.945222 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:33:37.945679 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:33:37.946135 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:33:37.946141 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:33:37.946145 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:33:37.946361 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:33:37.946796 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:33:37.946805 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:33:37.947213 (+ 408us) client_negotiation.cc:769] Sending connection context
0619 02:33:37.947459 (+ 246us) client_negotiation.cc:240] Negotiation successful
0619 02:33:37.947851 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":135,"threads_started":1}
WARNING: no policy specified for kudu/127.17.96.195@KRBTEST.COM; defaulting to no policy
I20260619 02:33:37.950433 2684 heartbeater.cc:347] Connected to a master server at 127.17.96.254:45455
I20260619 02:33:37.950660 2684 heartbeater.cc:464] Registering TS with master...
I20260619 02:33:37.951196 2684 heartbeater.cc:511] Master 127.17.96.254:45455 requested a full tablet report, sending...
I20260619 02:33:37.952286 2359 ts_manager.cc:194] Registered new tserver with Master: d70a1ba6a94e4c1eb4e130590ce5f106 (127.17.96.194:43947)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.195@KRBTEST.COM" created.
I20260619 02:33:37.953135 2359 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:47715
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
I20260619 02:33:37.968777 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45455
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:41971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:33:38.085552 2689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:38.085844 2689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:38.085937 2689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:38.090260 2689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:38.090338 2689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:38.090456 2689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:33:38.096143 2689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:41971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:38.097289 2689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:38.098392 2689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:38.108348 2695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:38.108376 2697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:38.108526 2689 server_base.cc:1034] running on GCE node
W20260619 02:33:38.108412 2694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:38.109138 2689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:38.109827 2689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:38.111043 2689 hybrid_clock.cc:648] HybridClock initialized: now 1781836418111010 us; error 51 us; skew 500 ppm
Jun 19 02:33:38 dist-test-slave-4qw6 krb5kdc[2314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836418, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:38.114003 2689 init.cc:377] Logged in from keytab as kudu/127.17.96.195@KRBTEST.COM (short username kudu)
I20260619 02:33:38.114889 2689 webserver.cc:460] Webserver started at http://127.17.96.195:40605/ using document root <none> and password file <none>
I20260619 02:33:38.115438 2689 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:38.115510 2689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:38.115687 2689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:38.117890 2689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "d7d3fc7d3eff4308bcfe19c0f56429be"
format_stamp: "Formatted at 2026-06-19 02:33:38 on dist-test-slave-4qw6"
server_key: "712a12e8629653623155b0f7b8049009"
server_key_iv: "0706bbefec42d3c3191ce257d710e534"
server_key_version: "encryptionkey@0"
I20260619 02:33:38.118467 2689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "d7d3fc7d3eff4308bcfe19c0f56429be"
format_stamp: "Formatted at 2026-06-19 02:33:38 on dist-test-slave-4qw6"
server_key: "712a12e8629653623155b0f7b8049009"
server_key_iv: "0706bbefec42d3c3191ce257d710e534"
server_key_version: "encryptionkey@0"
I20260619 02:33:38.122466 2689 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260619 02:33:38.125072 2704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:38.126482 2689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 02:33:38.126632 2689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "d7d3fc7d3eff4308bcfe19c0f56429be"
format_stamp: "Formatted at 2026-06-19 02:33:38 on dist-test-slave-4qw6"
server_key: "712a12e8629653623155b0f7b8049009"
server_key_iv: "0706bbefec42d3c3191ce257d710e534"
server_key_version: "encryptionkey@0"
I20260619 02:33:38.126754 2689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:38.139787 2689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:38.143455 2689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:38.143695 2689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:38.144279 2689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:33:38.145169 2689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:33:38.145217 2689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:38.145285 2689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:33:38.145321 2689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:38.155242 2689 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:43199
I20260619 02:33:38.155280 2817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:43199 every 8 connection(s)
I20260619 02:33:38.156268 2689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:33:38.165931 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 2689
I20260619 02:33:38.166114 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:33:38.166395 17795 external_mini_cluster.cc:1428] Setting key 5b0038c248bc79481b7f9add922eba23
Jun 19 02:33:38 dist-test-slave-4qw6 krb5kdc[2314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836418, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:33:38.171275 2398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:38.158198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:56325 (local address 127.17.96.254:45455)
0619 02:33:38.158359 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:38.158363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:38.159278 (+ 915us) server_negotiation.cc:407] Connection header received
0619 02:33:38.160719 (+ 1441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:38.160723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:38.160799 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:38.160909 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:38.162203 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.162757 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.163974 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.164209 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.164333 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:38.167231 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:38.167242 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:38.167245 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:33:38.167272 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:38.169064 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:38.169660 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:38.169663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:38.169664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:38.169796 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:38.170171 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:38.170174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:38.170175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:38.170366 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:38.170454 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:38.171009 (+ 555us) server_negotiation.cc:299] Negotiation successful
0619 02:33:38.171137 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 02:33:38.172099 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:38.158617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45455 (local address 127.17.96.195:56325)
0619 02:33:38.159126 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:38.159170 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:38.160480 (+ 1310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:38.161092 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:38.161100 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:38.161486 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:33:38.162036 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:38.162050 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.162901 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.162907 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:38.163833 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:38.163843 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.164378 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.164382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:38.164597 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:38.165187 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:33:38.165203 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:33:38.167037 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:33:38.169173 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:33:38.169178 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:33:38.169193 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:33:38.169552 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:33:38.169898 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:33:38.169900 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:33:38.169902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:33:38.170087 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:33:38.170455 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:33:38.170460 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:33:38.170745 (+ 285us) client_negotiation.cc:769] Sending connection context
0619 02:33:38.170920 (+ 175us) client_negotiation.cc:240] Negotiation successful
0619 02:33:38.171208 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":119,"threads_started":1}
I20260619 02:33:38.173264 2818 heartbeater.cc:347] Connected to a master server at 127.17.96.254:45455
I20260619 02:33:38.173524 2818 heartbeater.cc:464] Registering TS with master...
I20260619 02:33:38.174099 2818 heartbeater.cc:511] Master 127.17.96.254:45455 requested a full tablet report, sending...
I20260619 02:33:38.175305 2359 ts_manager.cc:194] Registered new tserver with Master: d7d3fc7d3eff4308bcfe19c0f56429be (127.17.96.195:43199)
I20260619 02:33:38.175910 2359 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:56325
I20260619 02:33:38.180737 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:33:38.190481 2398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:38.182867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51794 (local address 127.17.96.254:45455)
0619 02:33:38.183044 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:38.183049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:38.183149 (+ 100us) server_negotiation.cc:407] Connection header received
0619 02:33:38.183243 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:38.183246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:38.183299 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:38.183392 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:38.184109 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.184638 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.185492 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.185704 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.185867 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:38.187212 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:38.187215 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:38.187217 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:33:38.187242 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:38.188871 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:38.189246 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:38.189248 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:38.189249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:38.189322 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:38.189550 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:38.189552 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:38.189553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:38.189837 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:38.189941 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:38.190169 (+ 228us) server_negotiation.cc:299] Negotiation successful
0619 02:33:38.190269 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 02:33:38.193490 2359 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51794:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:33:38.196713 2359 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:33:38.213819 2830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:38.206084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:39739 (local address 127.0.0.1:39174)
0619 02:33:38.207623 (+ 1539us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:38.207637 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:38.207716 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:38.208333 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:38.208336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:38.208353 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:38.208546 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:38.208554 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.210233 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.210237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:38.211758 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:38.211769 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.213124 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.213127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:38.213257 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:38.213314 (+ 57us) client_negotiation.cc:769] Sending connection context
0619 02:33:38.213431 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 02:33:38.213574 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1474,"mutex_wait_us":65,"thread_start_us":91,"threads_started":1}
I20260619 02:33:38.214035 2834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:38.206857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36438 (local address 127.17.96.194:43947)
0619 02:33:38.207236 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:38.207246 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:38.208466 (+ 1220us) server_negotiation.cc:407] Connection header received
0619 02:33:38.208704 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:38.208709 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:38.208893 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:38.209013 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:38.209480 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.210837 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.212316 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.213159 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.213243 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:38.213392 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:38.213731 (+ 339us) server_negotiation.cc:299] Negotiation successful
0619 02:33:38.213899 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":97,"threads_started":1}
I20260619 02:33:38.214393 2829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:38.206428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:43947 (local address 127.0.0.1:36438)
0619 02:33:38.208373 (+ 1945us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:38.208386 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:38.208555 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:38.209205 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:38.209207 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:38.209224 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:38.209302 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:38.209306 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.211002 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.211005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:38.212186 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:38.212195 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.213481 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.213484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:38.213574 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:38.213595 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 02:33:38.214177 (+ 582us) client_negotiation.cc:240] Negotiation successful
0619 02:33:38.214264 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1873,"mutex_wait_us":1,"thread_start_us":88,"threads_started":1}
I20260619 02:33:38.215036 2832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:38.205662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:43199 (local address 127.0.0.1:55190)
0619 02:33:38.207293 (+ 1631us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:38.207364 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:38.207500 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:38.208149 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:38.208160 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:38.208192 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:38.208507 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:38.208519 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.211415 (+ 2896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.211420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:38.212855 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:38.212871 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.214647 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.214650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:38.214742 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:38.214776 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 02:33:38.214845 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 02:33:38.214914 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1444,"mutex_wait_us":1,"thread_start_us":119,"threads_started":1}
I20260619 02:33:38.215210 2831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:38.205972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55190 (local address 127.17.96.195:43199)
0619 02:33:38.207456 (+ 1484us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:38.207468 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:38.207492 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:33:38.207710 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:38.207716 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:38.207942 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:38.208098 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:38.208722 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.210415 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.213021 (+ 2606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.214058 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.214669 (+ 611us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:38.214855 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:38.214909 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 02:33:38.215022 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1376,"thread_start_us":304,"threads_started":1}
I20260619 02:33:38.215332 2833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:38.206525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39174 (local address 127.17.96.193:39739)
0619 02:33:38.207024 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:38.207040 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:38.207807 (+ 767us) server_negotiation.cc:407] Connection header received
0619 02:33:38.207895 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:38.207900 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:38.208060 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:38.208180 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:38.208680 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.209998 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.211899 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.212785 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.212893 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:38.213055 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:38.214253 (+ 1198us) server_negotiation.cc:299] Negotiation successful
0619 02:33:38.214366 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":107,"threads_started":1}
I20260619 02:33:38.217738 2752 tablet_service.cc:1467] Processing CreateTablet for tablet c810cbf360b74059b62e625142422da9 (DEFAULT_TABLE table=test-table [id=ddc108158a55486dbebab8d807f6a084]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:33:38.217738 2618 tablet_service.cc:1467] Processing CreateTablet for tablet c810cbf360b74059b62e625142422da9 (DEFAULT_TABLE table=test-table [id=ddc108158a55486dbebab8d807f6a084]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:33:38.218210 2484 tablet_service.cc:1467] Processing CreateTablet for tablet c810cbf360b74059b62e625142422da9 (DEFAULT_TABLE table=test-table [id=ddc108158a55486dbebab8d807f6a084]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:33:38.219015 2618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c810cbf360b74059b62e625142422da9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:38.219015 2752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c810cbf360b74059b62e625142422da9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:38.219224 2484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c810cbf360b74059b62e625142422da9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:38.226791 2835 tablet_bootstrap.cc:492] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9: Bootstrap starting.
I20260619 02:33:38.229251 2835 tablet_bootstrap.cc:654] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:38.230482 2835 log.cc:824] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:38.231348 2836 tablet_bootstrap.cc:492] T c810cbf360b74059b62e625142422da9 P d7d3fc7d3eff4308bcfe19c0f56429be: Bootstrap starting.
I20260619 02:33:38.233635 2836 tablet_bootstrap.cc:654] T c810cbf360b74059b62e625142422da9 P d7d3fc7d3eff4308bcfe19c0f56429be: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:38.234860 2836 log.cc:824] T c810cbf360b74059b62e625142422da9 P d7d3fc7d3eff4308bcfe19c0f56429be: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:38.235148 2837 tablet_bootstrap.cc:492] T c810cbf360b74059b62e625142422da9 P d70a1ba6a94e4c1eb4e130590ce5f106: Bootstrap starting.
I20260619 02:33:38.236613 2835 tablet_bootstrap.cc:492] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9: No bootstrap required, opened a new log
I20260619 02:33:38.236869 2835 ts_tablet_manager.cc:1397] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260619 02:33:38.237848 2837 tablet_bootstrap.cc:654] T c810cbf360b74059b62e625142422da9 P d70a1ba6a94e4c1eb4e130590ce5f106: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:38.239259 2837 log.cc:824] T c810cbf360b74059b62e625142422da9 P d70a1ba6a94e4c1eb4e130590ce5f106: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:38.240326 2835 raft_consensus.cc:348] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d3fc7d3eff4308bcfe19c0f56429be" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 43199 } } peers { permanent_uuid: "d70a1ba6a94e4c1eb4e130590ce5f106" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43947 } } peers { permanent_uuid: "826d070a24b149729b184ac7a161b0b9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 39739 } }
I20260619 02:33:38.240588 2835 raft_consensus.cc:374] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:38.240648 2835 raft_consensus.cc:729] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 826d070a24b149729b184ac7a161b0b9, State: Initialized, Role: FOLLOWER
I20260619 02:33:38.241115 2835 consensus_queue.cc:260] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d3fc7d3eff4308bcfe19c0f56429be" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 43199 } } peers { permanent_uuid: "d70a1ba6a94e4c1eb4e130590ce5f106" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43947 } } peers { permanent_uuid: "826d070a24b149729b184ac7a161b0b9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 39739 } }
I20260619 02:33:38.241830 2550 heartbeater.cc:503] Master 127.17.96.254:45455 was elected leader, sending a full tablet report...
I20260619 02:33:38.242210 2835 ts_tablet_manager.cc:1428] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 02:33:38.242719 2837 tablet_bootstrap.cc:492] T c810cbf360b74059b62e625142422da9 P d70a1ba6a94e4c1eb4e130590ce5f106: No bootstrap required, opened a new log
I20260619 02:33:38.242906 2837 ts_tablet_manager.cc:1397] T c810cbf360b74059b62e625142422da9 P d70a1ba6a94e4c1eb4e130590ce5f106: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260619 02:33:38.243620 2836 tablet_bootstrap.cc:492] T c810cbf360b74059b62e625142422da9 P d7d3fc7d3eff4308bcfe19c0f56429be: No bootstrap required, opened a new log
I20260619 02:33:38.243793 2836 ts_tablet_manager.cc:1397] T c810cbf360b74059b62e625142422da9 P d7d3fc7d3eff4308bcfe19c0f56429be: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20260619 02:33:38.247732 2836 raft_consensus.cc:348] T c810cbf360b74059b62e625142422da9 P d7d3fc7d3eff4308bcfe19c0f56429be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d3fc7d3eff4308bcfe19c0f56429be" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 43199 } } peers { permanent_uuid: "d70a1ba6a94e4c1eb4e130590ce5f106" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43947 } } peers { permanent_uuid: "826d070a24b149729b184ac7a161b0b9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 39739 } }
I20260619 02:33:38.248025 2836 raft_consensus.cc:374] T c810cbf360b74059b62e625142422da9 P d7d3fc7d3eff4308bcfe19c0f56429be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:38.248081 2836 raft_consensus.cc:729] T c810cbf360b74059b62e625142422da9 P d7d3fc7d3eff4308bcfe19c0f56429be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7d3fc7d3eff4308bcfe19c0f56429be, State: Initialized, Role: FOLLOWER
I20260619 02:33:38.247972 2837 raft_consensus.cc:348] T c810cbf360b74059b62e625142422da9 P d70a1ba6a94e4c1eb4e130590ce5f106 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d3fc7d3eff4308bcfe19c0f56429be" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 43199 } } peers { permanent_uuid: "d70a1ba6a94e4c1eb4e130590ce5f106" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43947 } } peers { permanent_uuid: "826d070a24b149729b184ac7a161b0b9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 39739 } }
I20260619 02:33:38.248262 2837 raft_consensus.cc:374] T c810cbf360b74059b62e625142422da9 P d70a1ba6a94e4c1eb4e130590ce5f106 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:38.248335 2837 raft_consensus.cc:729] T c810cbf360b74059b62e625142422da9 P d70a1ba6a94e4c1eb4e130590ce5f106 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d70a1ba6a94e4c1eb4e130590ce5f106, State: Initialized, Role: FOLLOWER
I20260619 02:33:38.248629 2836 consensus_queue.cc:260] T c810cbf360b74059b62e625142422da9 P d7d3fc7d3eff4308bcfe19c0f56429be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d3fc7d3eff4308bcfe19c0f56429be" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 43199 } } peers { permanent_uuid: "d70a1ba6a94e4c1eb4e130590ce5f106" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43947 } } peers { permanent_uuid: "826d070a24b149729b184ac7a161b0b9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 39739 } }
I20260619 02:33:38.248798 2837 consensus_queue.cc:260] T c810cbf360b74059b62e625142422da9 P d70a1ba6a94e4c1eb4e130590ce5f106 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d3fc7d3eff4308bcfe19c0f56429be" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 43199 } } peers { permanent_uuid: "d70a1ba6a94e4c1eb4e130590ce5f106" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43947 } } peers { permanent_uuid: "826d070a24b149729b184ac7a161b0b9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 39739 } }
I20260619 02:33:38.249575 2684 heartbeater.cc:503] Master 127.17.96.254:45455 was elected leader, sending a full tablet report...
I20260619 02:33:38.249841 2836 ts_tablet_manager.cc:1428] T c810cbf360b74059b62e625142422da9 P d7d3fc7d3eff4308bcfe19c0f56429be: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260619 02:33:38.249919 2837 ts_tablet_manager.cc:1428] T c810cbf360b74059b62e625142422da9 P d70a1ba6a94e4c1eb4e130590ce5f106: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260619 02:33:38.250171 2818 heartbeater.cc:503] Master 127.17.96.254:45455 was elected leader, sending a full tablet report...
I20260619 02:33:38.330687 2841 raft_consensus.cc:482] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:33:38.330925 2841 raft_consensus.cc:504] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d3fc7d3eff4308bcfe19c0f56429be" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 43199 } } peers { permanent_uuid: "d70a1ba6a94e4c1eb4e130590ce5f106" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43947 } } peers { permanent_uuid: "826d070a24b149729b184ac7a161b0b9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 39739 } }
I20260619 02:33:38.332273 2841 leader_election.cc:288] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7d3fc7d3eff4308bcfe19c0f56429be (127.17.96.195:43199), d70a1ba6a94e4c1eb4e130590ce5f106 (127.17.96.194:43947)
I20260619 02:33:38.336308 2831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:38.332634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:34129 (local address 127.17.96.195:43199)
0619 02:33:38.332810 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:38.332815 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:38.332987 (+ 172us) server_negotiation.cc:407] Connection header received
0619 02:33:38.333164 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:38.333169 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:38.333234 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:38.333280 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:38.333671 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.334240 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.335303 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.335740 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.335859 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:38.335916 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:38.336098 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 02:33:38.336172 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 02:33:38.336323 2844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:38.332552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:43199 (local address 127.17.96.193:34129)
0619 02:33:38.332862 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:38.332889 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:38.333009 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:38.333423 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:38.333425 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:38.333445 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:38.333551 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:38.333555 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.334354 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.334360 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:38.335169 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:38.335179 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.335860 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.335863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:38.335973 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:38.335991 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 02:33:38.336082 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 02:33:38.336159 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":138,"threads_started":1}
I20260619 02:33:38.336550 2834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:38.333160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:55361 (local address 127.17.96.194:43947)
0619 02:33:38.333289 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:38.333294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:38.333423 (+ 129us) server_negotiation.cc:407] Connection header received
0619 02:33:38.333608 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:38.333613 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:38.333663 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:38.333756 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:38.334047 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.334530 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.335582 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.335995 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.336127 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:38.336178 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:38.336363 (+ 185us) server_negotiation.cc:299] Negotiation successful
0619 02:33:38.336438 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 02:33:38.336580 2845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:38.332993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:43947 (local address 127.17.96.193:55361)
0619 02:33:38.333310 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:38.333326 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:38.333478 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:38.333784 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:38.333787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:38.333804 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:38.333927 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:38.333933 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.334649 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.334653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:38.335452 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:38.335461 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.336120 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.336124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:38.336220 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:38.336241 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 02:33:38.336346 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 02:33:38.336416 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"spinlock_wait_cycles":30464,"thread_start_us":87,"threads_started":1}
I20260619 02:33:38.337126 2772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c810cbf360b74059b62e625142422da9" candidate_uuid: "826d070a24b149729b184ac7a161b0b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7d3fc7d3eff4308bcfe19c0f56429be" is_pre_election: true
I20260619 02:33:38.337159 2638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c810cbf360b74059b62e625142422da9" candidate_uuid: "826d070a24b149729b184ac7a161b0b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d70a1ba6a94e4c1eb4e130590ce5f106" is_pre_election: true
I20260619 02:33:38.337450 2638 raft_consensus.cc:2454] T c810cbf360b74059b62e625142422da9 P d70a1ba6a94e4c1eb4e130590ce5f106 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 826d070a24b149729b184ac7a161b0b9 in term 0.
I20260619 02:33:38.337452 2772 raft_consensus.cc:2454] T c810cbf360b74059b62e625142422da9 P d7d3fc7d3eff4308bcfe19c0f56429be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 826d070a24b149729b184ac7a161b0b9 in term 0.
I20260619 02:33:38.338004 2438 leader_election.cc:302] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 826d070a24b149729b184ac7a161b0b9, d70a1ba6a94e4c1eb4e130590ce5f106; no voters:
I20260619 02:33:38.338275 2841 raft_consensus.cc:2781] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:33:38.338357 2841 raft_consensus.cc:482] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:33:38.338392 2841 raft_consensus.cc:3037] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:38.339500 2841 raft_consensus.cc:504] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d3fc7d3eff4308bcfe19c0f56429be" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 43199 } } peers { permanent_uuid: "d70a1ba6a94e4c1eb4e130590ce5f106" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43947 } } peers { permanent_uuid: "826d070a24b149729b184ac7a161b0b9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 39739 } }
I20260619 02:33:38.339872 2841 leader_election.cc:288] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9 [CANDIDATE]: Term 1 election: Requested vote from peers d7d3fc7d3eff4308bcfe19c0f56429be (127.17.96.195:43199), d70a1ba6a94e4c1eb4e130590ce5f106 (127.17.96.194:43947)
I20260619 02:33:38.340370 2772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c810cbf360b74059b62e625142422da9" candidate_uuid: "826d070a24b149729b184ac7a161b0b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7d3fc7d3eff4308bcfe19c0f56429be"
I20260619 02:33:38.340461 2638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c810cbf360b74059b62e625142422da9" candidate_uuid: "826d070a24b149729b184ac7a161b0b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d70a1ba6a94e4c1eb4e130590ce5f106"
I20260619 02:33:38.340530 2772 raft_consensus.cc:3037] T c810cbf360b74059b62e625142422da9 P d7d3fc7d3eff4308bcfe19c0f56429be [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:38.340654 2638 raft_consensus.cc:3037] T c810cbf360b74059b62e625142422da9 P d70a1ba6a94e4c1eb4e130590ce5f106 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:38.341658 2772 raft_consensus.cc:2454] T c810cbf360b74059b62e625142422da9 P d7d3fc7d3eff4308bcfe19c0f56429be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 826d070a24b149729b184ac7a161b0b9 in term 1.
I20260619 02:33:38.341737 2638 raft_consensus.cc:2454] T c810cbf360b74059b62e625142422da9 P d70a1ba6a94e4c1eb4e130590ce5f106 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 826d070a24b149729b184ac7a161b0b9 in term 1.
I20260619 02:33:38.342041 2438 leader_election.cc:302] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 826d070a24b149729b184ac7a161b0b9, d7d3fc7d3eff4308bcfe19c0f56429be; no voters:
I20260619 02:33:38.342281 2841 raft_consensus.cc:2781] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:33:38.342519 2841 raft_consensus.cc:686] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9 [term 1 LEADER]: Becoming Leader. State: Replica: 826d070a24b149729b184ac7a161b0b9, State: Running, Role: LEADER
I20260619 02:33:38.342921 2841 consensus_queue.cc:237] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d3fc7d3eff4308bcfe19c0f56429be" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 43199 } } peers { permanent_uuid: "d70a1ba6a94e4c1eb4e130590ce5f106" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43947 } } peers { permanent_uuid: "826d070a24b149729b184ac7a161b0b9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 39739 } }
I20260619 02:33:38.346303 2357 catalog_manager.cc:5495] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 826d070a24b149729b184ac7a161b0b9 (127.17.96.193). New cstate: current_term: 1 leader_uuid: "826d070a24b149729b184ac7a161b0b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d3fc7d3eff4308bcfe19c0f56429be" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 43199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d70a1ba6a94e4c1eb4e130590ce5f106" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "826d070a24b149729b184ac7a161b0b9" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 39739 } health_report { overall_health: HEALTHY } } }
I20260619 02:33:38.384313 2833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:38.380456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39190 (local address 127.17.96.193:39739)
0619 02:33:38.380625 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:38.380631 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:38.380649 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:33:38.380750 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:38.380753 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:38.380811 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:38.380934 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:33:38.381221 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.381881 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.382747 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.382995 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.383111 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:38.383444 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:33:38.383909 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:33:38.384052 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:38.384098 (+ 46us) server_negotiation.cc:299] Negotiation successful
0619 02:33:38.384164 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 02:33:38.391021 2772 raft_consensus.cc:1261] T c810cbf360b74059b62e625142422da9 P d7d3fc7d3eff4308bcfe19c0f56429be [term 1 FOLLOWER]: Refusing update from remote peer 826d070a24b149729b184ac7a161b0b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:33:38.391024 2638 raft_consensus.cc:1261] T c810cbf360b74059b62e625142422da9 P d70a1ba6a94e4c1eb4e130590ce5f106 [term 1 FOLLOWER]: Refusing update from remote peer 826d070a24b149729b184ac7a161b0b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:33:38.391825 2841 consensus_queue.cc:1035] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d70a1ba6a94e4c1eb4e130590ce5f106" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:33:38.392067 2846 consensus_queue.cc:1035] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7d3fc7d3eff4308bcfe19c0f56429be" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 43199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:33:38.400115 2851 mvcc.cc:205] Tried to move back new op lower bound from 7298401969721298944 to 7298401969535430656. Current Snapshot: MvccSnapshot[applied={T|T < 7298401969721298944}]
I20260619 02:33:38.409789 2357 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:33:38.410018 2357 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:33:38.412947 2357 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d4252aca2b354406999e3ab7c6a30968: Sending DeleteTablet for 3 replicas of tablet c810cbf360b74059b62e625142422da9
I20260619 02:33:38.413976 2484 tablet_service.cc:1514] Processing DeleteTablet for tablet c810cbf360b74059b62e625142422da9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:33:38 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:39174
I20260619 02:33:38.413980 2752 tablet_service.cc:1514] Processing DeleteTablet for tablet c810cbf360b74059b62e625142422da9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:33:38 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:55190
I20260619 02:33:38.414532 2859 tablet_replica.cc:330] T c810cbf360b74059b62e625142422da9 P 826d070a24b149729b184ac7a161b0b9: stopping tablet replica
I20260619 02:33:38.414815 2618 tablet_service.cc:1514] Processing DeleteTablet for tablet c810cbf360b74059b62e625142422da9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:33:38 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:36438
I20260619 02:33:38.414986 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 2421
I20260619 02:33:38.416514 2860 tablet_replica.cc:330] T c810cbf360b74059b62e625142422da9 P d7d3fc7d3eff4308bcfe19c0f56429be: stopping tablet replica
I20260619 02:33:38.416888 2860 raft_consensus.cc:2229] T c810cbf360b74059b62e625142422da9 P d7d3fc7d3eff4308bcfe19c0f56429be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:33:38.417249 2860 raft_consensus.cc:2258] T c810cbf360b74059b62e625142422da9 P d7d3fc7d3eff4308bcfe19c0f56429be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:33:38.417637 2861 tablet_replica.cc:330] T c810cbf360b74059b62e625142422da9 P d70a1ba6a94e4c1eb4e130590ce5f106: stopping tablet replica
I20260619 02:33:38.417975 2861 raft_consensus.cc:2229] T c810cbf360b74059b62e625142422da9 P d70a1ba6a94e4c1eb4e130590ce5f106 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:33:38.418272 2861 raft_consensus.cc:2258] T c810cbf360b74059b62e625142422da9 P d70a1ba6a94e4c1eb4e130590ce5f106 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:33:38.419072 2860 ts_tablet_manager.cc:1905] T c810cbf360b74059b62e625142422da9 P d7d3fc7d3eff4308bcfe19c0f56429be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:33:38.419622 2861 ts_tablet_manager.cc:1905] T c810cbf360b74059b62e625142422da9 P d70a1ba6a94e4c1eb4e130590ce5f106: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:33:38.422152 2860 ts_tablet_manager.cc:1918] T c810cbf360b74059b62e625142422da9 P d7d3fc7d3eff4308bcfe19c0f56429be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:33:38.422271 2860 log.cc:1196] T c810cbf360b74059b62e625142422da9 P d7d3fc7d3eff4308bcfe19c0f56429be: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/c810cbf360b74059b62e625142422da9
I20260619 02:33:38.422504 2861 ts_tablet_manager.cc:1918] T c810cbf360b74059b62e625142422da9 P d70a1ba6a94e4c1eb4e130590ce5f106: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:33:38.422608 2861 log.cc:1196] T c810cbf360b74059b62e625142422da9 P d70a1ba6a94e4c1eb4e130590ce5f106: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/c810cbf360b74059b62e625142422da9
I20260619 02:33:38.422647 2860 ts_tablet_manager.cc:1939] T c810cbf360b74059b62e625142422da9 P d7d3fc7d3eff4308bcfe19c0f56429be: Deleting consensus metadata
W20260619 02:33:38.422878 2346 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.96.193:39739 (error 108)
I20260619 02:33:38.422922 2861 ts_tablet_manager.cc:1939] T c810cbf360b74059b62e625142422da9 P d70a1ba6a94e4c1eb4e130590ce5f106: Deleting consensus metadata
I20260619 02:33:38.423733 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 2555
I20260619 02:33:38.424731 2344 catalog_manager.cc:4841] TS d7d3fc7d3eff4308bcfe19c0f56429be (127.17.96.195:43199): tablet c810cbf360b74059b62e625142422da9 (table test-table [id=ddc108158a55486dbebab8d807f6a084]) successfully deleted
I20260619 02:33:38.430459 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 2689
I20260619 02:33:38.430635 2832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:38.427577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:39739 (local address 127.0.0.1:39194)
0619 02:33:38.430445 (+ 2868us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:38.430534 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.193:39739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2745}
W20260619 02:33:38.432435 2346 catalog_manager.cc:4584] TS 826d070a24b149729b184ac7a161b0b9 (127.17.96.193:39739): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c810cbf360b74059b62e625142422da9: Network error: Client connection negotiation failed: client connection to 127.17.96.193:39739: connect: Connection refused (error 111)
I20260619 02:33:38.433197 2832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:38.432942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:43947 (local address 127.0.0.1:36444)
0619 02:33:38.433072 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:38.433132 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.194:43947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 02:33:38.437379 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 2327
2026-06-19T02:33:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2423 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-19T02:33:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:33:38Z Disabled control of system clock
I20260619 02:33:38.468894 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:46629
--webserver_interface=127.17.96.254
--webserver_port=0
--builtin_ntp_servers=127.17.96.212:39021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:46629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:33:38.585245 2866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:38.585543 2866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:38.585580 2866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:38.589956 2866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:33:38.590035 2866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:38.590051 2866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:38.590093 2866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:33:38.590126 2866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:33:38.595456 2866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:39021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:46629
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:46629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:38.597194 2866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:38.599396 2866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:38.609127 2871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:38.609187 2872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:38.609144 2874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:38.609422 2866 server_base.cc:1034] running on GCE node
I20260619 02:33:38.610107 2866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:38.611243 2866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:38.612418 2866 hybrid_clock.cc:648] HybridClock initialized: now 1781836418612407 us; error 32 us; skew 500 ppm
I20260619 02:33:38.614477 2866 webserver.cc:460] Webserver started at http://127.17.96.254:46193/ using document root <none> and password file <none>
I20260619 02:33:38.615108 2866 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:38.615176 2866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:38.615432 2866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:38.617645 2866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "91c074ee4b8a47d9be5d473ee6cd88bd"
format_stamp: "Formatted at 2026-06-19 02:33:38 on dist-test-slave-4qw6"
server_key: "dcb8128172a7fffd788029ceedaabe54"
server_key_iv: "f50b0c892103e1b597ffe05ededfec11"
server_key_version: "encryptionkey@0"
I20260619 02:33:38.618333 2866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "91c074ee4b8a47d9be5d473ee6cd88bd"
format_stamp: "Formatted at 2026-06-19 02:33:38 on dist-test-slave-4qw6"
server_key: "dcb8128172a7fffd788029ceedaabe54"
server_key_iv: "f50b0c892103e1b597ffe05ededfec11"
server_key_version: "encryptionkey@0"
I20260619 02:33:38.622447 2866 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 02:33:38.625087 2880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:38.626498 2866 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:33:38.626667 2866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "91c074ee4b8a47d9be5d473ee6cd88bd"
format_stamp: "Formatted at 2026-06-19 02:33:38 on dist-test-slave-4qw6"
server_key: "dcb8128172a7fffd788029ceedaabe54"
server_key_iv: "f50b0c892103e1b597ffe05ededfec11"
server_key_version: "encryptionkey@0"
I20260619 02:33:38.626806 2866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:38.645979 2866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:38.646775 2866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:38.647001 2866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:38.654325 2866 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:46629
I20260619 02:33:38.654330 2932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:46629 every 8 connection(s)
I20260619 02:33:38.655488 2866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:33:38.658758 2933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:38.664808 2933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd: Bootstrap starting.
I20260619 02:33:38.665709 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 2866
I20260619 02:33:38.665920 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:33:38.666201 17795 external_mini_cluster.cc:1428] Setting key f69238ab588dd5d752aa03e4c780947e
I20260619 02:33:38.668040 2933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:38.668850 2933 log.cc:824] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:38.671087 2933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd: No bootstrap required, opened a new log
I20260619 02:33:38.675113 2933 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c074ee4b8a47d9be5d473ee6cd88bd" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46629 } }
I20260619 02:33:38.675419 2933 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:38.675506 2933 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c074ee4b8a47d9be5d473ee6cd88bd, State: Initialized, Role: FOLLOWER
I20260619 02:33:38.676008 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:38.667472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34294 (local address 127.17.96.254:46629)
0619 02:33:38.668197 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:38.668218 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:38.668260 (+ 42us) server_negotiation.cc:407] Connection header received
0619 02:33:38.668955 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:38.668975 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:38.669447 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:38.669818 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:38.670276 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.671450 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.672388 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.672896 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.673080 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:38.674089 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:38.674094 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:38.674108 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:38.674531 (+ 423us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:38.674588 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:38.674609 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:38.674721 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:38.674839 (+ 118us) server_negotiation.cc:299] Negotiation successful
0619 02:33:38.675122 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":143,"threads_started":1}
I20260619 02:33:38.676019 2933 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c074ee4b8a47d9be5d473ee6cd88bd" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46629 } }
I20260619 02:33:38.676278 2933 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:33:38.676342 2933 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:33:38.676431 2933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:38.677599 2933 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c074ee4b8a47d9be5d473ee6cd88bd" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46629 } }
I20260619 02:33:38.678001 2933 leader_election.cc:302] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91c074ee4b8a47d9be5d473ee6cd88bd; no voters:
I20260619 02:33:38.678403 2933 leader_election.cc:288] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:33:38.678514 2938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:33:38.678748 2938 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd [term 1 LEADER]: Becoming Leader. State: Replica: 91c074ee4b8a47d9be5d473ee6cd88bd, State: Running, Role: LEADER
I20260619 02:33:38.679108 2938 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c074ee4b8a47d9be5d473ee6cd88bd" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46629 } }
I20260619 02:33:38.679668 2933 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:33:38.680853 2940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91c074ee4b8a47d9be5d473ee6cd88bd. Latest consensus state: current_term: 1 leader_uuid: "91c074ee4b8a47d9be5d473ee6cd88bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c074ee4b8a47d9be5d473ee6cd88bd" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46629 } } }
I20260619 02:33:38.681005 2940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:38.680858 2939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91c074ee4b8a47d9be5d473ee6cd88bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c074ee4b8a47d9be5d473ee6cd88bd" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46629 } } }
I20260619 02:33:38.681113 2939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:38.681435 2947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:33:38.684998 2947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:33:38.691375 2947 catalog_manager.cc:1347] Generated new cluster ID: 83b962768fde4412981f5f416bdeaf2e
I20260619 02:33:38.691489 2947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:33:38.712024 2947 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:33:38.713079 2947 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:33:38.726439 2947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd: Generated new TSK 0
I20260619 02:33:38.727298 2947 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:33:38.731591 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:46629
--builtin_ntp_servers=127.17.96.212:39021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:33:38.864619 2957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:38.864923 2957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:38.864964 2957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:38.869345 2957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:38.869414 2957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:38.869481 2957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:33:38.874701 2957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:39021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:46629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:38.875963 2957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:38.877441 2957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:38.888413 2963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:38.888540 2965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:38.888655 2957 server_base.cc:1034] running on GCE node
W20260619 02:33:38.888535 2962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:38.889151 2957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:38.889921 2957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:38.891096 2957 hybrid_clock.cc:648] HybridClock initialized: now 1781836418891082 us; error 30 us; skew 500 ppm
I20260619 02:33:38.893013 2957 webserver.cc:460] Webserver started at http://127.17.96.193:42959/ using document root <none> and password file <none>
I20260619 02:33:38.893606 2957 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:38.893673 2957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:38.893936 2957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:38.896242 2957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "c1bd7c5629de4b87bceb799b7fbefc57"
format_stamp: "Formatted at 2026-06-19 02:33:38 on dist-test-slave-4qw6"
server_key: "493a6c113ce54039663250fa2626a0bc"
server_key_iv: "0c92475256c828732745387cef7d1e4d"
server_key_version: "encryptionkey@0"
I20260619 02:33:38.896792 2957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "c1bd7c5629de4b87bceb799b7fbefc57"
format_stamp: "Formatted at 2026-06-19 02:33:38 on dist-test-slave-4qw6"
server_key: "493a6c113ce54039663250fa2626a0bc"
server_key_iv: "0c92475256c828732745387cef7d1e4d"
server_key_version: "encryptionkey@0"
I20260619 02:33:38.900903 2957 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 02:33:38.903645 2971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:38.905011 2957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:33:38.905128 2957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "c1bd7c5629de4b87bceb799b7fbefc57"
format_stamp: "Formatted at 2026-06-19 02:33:38 on dist-test-slave-4qw6"
server_key: "493a6c113ce54039663250fa2626a0bc"
server_key_iv: "0c92475256c828732745387cef7d1e4d"
server_key_version: "encryptionkey@0"
I20260619 02:33:38.905237 2957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:38.919118 2957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:38.919917 2957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:38.920089 2957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:38.920655 2957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:33:38.921608 2957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:33:38.921656 2957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:38.921700 2957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:33:38.921723 2957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:38.931319 2957 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:46129
I20260619 02:33:38.931345 3084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:46129 every 8 connection(s)
I20260619 02:33:38.932382 2957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:33:38.939916 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 2957
I20260619 02:33:38.940076 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:33:38.940428 17795 external_mini_cluster.cc:1428] Setting key 6310463b16cf6a134c187ad00c0c8a96
I20260619 02:33:38.942574 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:38.934377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:52297 (local address 127.17.96.254:46629)
0619 02:33:38.934523 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:38.934526 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:38.935625 (+ 1099us) server_negotiation.cc:407] Connection header received
0619 02:33:38.937024 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:38.937027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:38.937080 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:38.937161 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:38.938209 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.938765 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.940184 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.940438 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.940573 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:38.942011 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:38.942016 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:38.942018 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:38.942032 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:38.942057 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:38.942068 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:38.942142 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:38.942316 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 02:33:38.942385 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 02:33:38.942734 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:46629
--builtin_ntp_servers=127.17.96.212:39021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 02:33:38.943513 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:38.934791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:46629 (local address 127.17.96.193:52297)
0619 02:33:38.935464 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:38.935507 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:38.936804 (+ 1297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:38.937338 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:38.937344 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:38.937433 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:33:38.938057 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:38.938073 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.938884 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.938886 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:38.939981 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:38.939995 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:38.940676 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:38.940682 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:38.940903 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:38.941621 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:33:38.941623 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:33:38.941669 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:33:38.941782 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:33:38.942159 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:33:38.942168 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:33:38.942173 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:33:38.942289 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 02:33:38.942552 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":117,"threads_started":1}
I20260619 02:33:38.944893 3085 heartbeater.cc:347] Connected to a master server at 127.17.96.254:46629
I20260619 02:33:38.945191 3085 heartbeater.cc:464] Registering TS with master...
I20260619 02:33:38.945876 3085 heartbeater.cc:511] Master 127.17.96.254:46629 requested a full tablet report, sending...
I20260619 02:33:38.947571 2897 ts_manager.cc:194] Registered new tserver with Master: c1bd7c5629de4b87bceb799b7fbefc57 (127.17.96.193:46129)
I20260619 02:33:38.948807 2897 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.96.193:52297
W20260619 02:33:39.059361 3088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:39.059710 3088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:39.059792 3088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:39.064190 3088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:39.064311 3088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:39.064409 3088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:33:39.071072 3088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:39021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:46629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:39.072420 3088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:39.074002 3088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:39.084494 3094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:39.084488 3093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:39.084494 3096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:39.084821 3088 server_base.cc:1034] running on GCE node
I20260619 02:33:39.085337 3088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:39.086123 3088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:39.087324 3088 hybrid_clock.cc:648] HybridClock initialized: now 1781836419087285 us; error 58 us; skew 500 ppm
I20260619 02:33:39.089282 3088 webserver.cc:460] Webserver started at http://127.17.96.194:38103/ using document root <none> and password file <none>
I20260619 02:33:39.090057 3088 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:39.090160 3088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:39.090408 3088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:39.092635 3088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "cdf6fbed6bc14b47a68c89990f439a2a"
format_stamp: "Formatted at 2026-06-19 02:33:39 on dist-test-slave-4qw6"
server_key: "83a1346fc53eba521ec5043a28a75891"
server_key_iv: "189d7023b08df8cd1863c40d9aa457ec"
server_key_version: "encryptionkey@0"
I20260619 02:33:39.093272 3088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "cdf6fbed6bc14b47a68c89990f439a2a"
format_stamp: "Formatted at 2026-06-19 02:33:39 on dist-test-slave-4qw6"
server_key: "83a1346fc53eba521ec5043a28a75891"
server_key_iv: "189d7023b08df8cd1863c40d9aa457ec"
server_key_version: "encryptionkey@0"
I20260619 02:33:39.097328 3088 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 02:33:39.100119 3102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:39.101454 3088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:33:39.101611 3088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "cdf6fbed6bc14b47a68c89990f439a2a"
format_stamp: "Formatted at 2026-06-19 02:33:39 on dist-test-slave-4qw6"
server_key: "83a1346fc53eba521ec5043a28a75891"
server_key_iv: "189d7023b08df8cd1863c40d9aa457ec"
server_key_version: "encryptionkey@0"
I20260619 02:33:39.101827 3088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:39.141798 3088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:39.142592 3088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:39.142812 3088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:39.143443 3088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:33:39.144457 3088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:33:39.144531 3088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:39.144620 3088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:33:39.144667 3088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:39.154496 3088 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:38841
I20260619 02:33:39.154539 3215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:38841 every 8 connection(s)
I20260619 02:33:39.155485 3088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:33:39.161073 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 3088
I20260619 02:33:39.161191 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:33:39.161468 17795 external_mini_cluster.cc:1428] Setting key a98b1e45ef14907834ef2e10028d72bb
I20260619 02:33:39.163750 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:46629
--builtin_ntp_servers=127.17.96.212:39021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 02:33:39.166755 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:39.157594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:40415 (local address 127.17.96.254:46629)
0619 02:33:39.157781 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:39.157785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:39.158950 (+ 1165us) server_negotiation.cc:407] Connection header received
0619 02:33:39.160484 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:39.160488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:39.160555 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:39.160674 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:39.161807 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.162667 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.164157 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.164609 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.164758 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:39.166226 (+ 1468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:39.166236 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:39.166239 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:39.166254 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:39.166274 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:39.166282 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:39.166370 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:39.166565 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 02:33:39.166633 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 02:33:39.168109 3218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:39.158059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:46629 (local address 127.17.96.194:40415)
0619 02:33:39.158760 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:39.158810 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:39.160221 (+ 1411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:39.160872 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:39.160880 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:39.160973 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:33:39.161621 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:39.161637 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.162853 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.162859 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:39.164005 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:39.164020 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.164910 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.164913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:39.165110 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:39.165897 (+ 787us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:33:39.165899 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:33:39.165953 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:33:39.166034 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:33:39.166372 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:33:39.166385 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:33:39.166395 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 02:33:39.166544 (+ 149us) client_negotiation.cc:240] Negotiation successful
0619 02:33:39.166902 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":133,"threads_started":1}
I20260619 02:33:39.169478 3216 heartbeater.cc:347] Connected to a master server at 127.17.96.254:46629
I20260619 02:33:39.169854 3216 heartbeater.cc:464] Registering TS with master...
I20260619 02:33:39.170486 3216 heartbeater.cc:511] Master 127.17.96.254:46629 requested a full tablet report, sending...
I20260619 02:33:39.171737 2897 ts_manager.cc:194] Registered new tserver with Master: cdf6fbed6bc14b47a68c89990f439a2a (127.17.96.194:38841)
I20260619 02:33:39.172390 2897 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.96.194:40415
W20260619 02:33:39.283406 3219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:39.283753 3219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:39.283829 3219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:39.288187 3219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:39.288311 3219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:39.288416 3219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:33:39.293677 3219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:39021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:46629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:39.294931 3219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:39.296377 3219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:39.306823 3225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:39.306878 3227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:39.306821 3224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:39.307273 3219 server_base.cc:1034] running on GCE node
I20260619 02:33:39.307755 3219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:39.308543 3219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:39.309737 3219 hybrid_clock.cc:648] HybridClock initialized: now 1781836419309706 us; error 62 us; skew 500 ppm
I20260619 02:33:39.311988 3219 webserver.cc:460] Webserver started at http://127.17.96.195:43975/ using document root <none> and password file <none>
I20260619 02:33:39.312633 3219 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:39.312738 3219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:39.312988 3219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:39.315263 3219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "36ce49f31df940be9a48068c6d28852f"
format_stamp: "Formatted at 2026-06-19 02:33:39 on dist-test-slave-4qw6"
server_key: "4eb1a4950fe298a7d82df820931ec4e6"
server_key_iv: "0eb8d195de617311015c43782aa11824"
server_key_version: "encryptionkey@0"
I20260619 02:33:39.315871 3219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "36ce49f31df940be9a48068c6d28852f"
format_stamp: "Formatted at 2026-06-19 02:33:39 on dist-test-slave-4qw6"
server_key: "4eb1a4950fe298a7d82df820931ec4e6"
server_key_iv: "0eb8d195de617311015c43782aa11824"
server_key_version: "encryptionkey@0"
I20260619 02:33:39.320012 3219 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:33:39.322865 3233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:39.324280 3219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:33:39.324474 3219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "36ce49f31df940be9a48068c6d28852f"
format_stamp: "Formatted at 2026-06-19 02:33:39 on dist-test-slave-4qw6"
server_key: "4eb1a4950fe298a7d82df820931ec4e6"
server_key_iv: "0eb8d195de617311015c43782aa11824"
server_key_version: "encryptionkey@0"
I20260619 02:33:39.324678 3219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:39.342532 3219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:39.343356 3219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:39.343587 3219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:39.344225 3219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:33:39.345265 3219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:33:39.345346 3219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:39.345424 3219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:33:39.345468 3219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:39.354723 3219 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:41791
I20260619 02:33:39.354754 3346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:41791 every 8 connection(s)
I20260619 02:33:39.355679 3219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:33:39.361792 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 3219
I20260619 02:33:39.361943 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:33:39.362248 17795 external_mini_cluster.cc:1428] Setting key 649b8ebf25c8b28df207d20ab934eecc
I20260619 02:33:39.366047 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:39.357811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:40671 (local address 127.17.96.254:46629)
0619 02:33:39.357973 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:39.357977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:39.359091 (+ 1114us) server_negotiation.cc:407] Connection header received
0619 02:33:39.360677 (+ 1586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:39.360681 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:39.360748 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:39.360889 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:39.361966 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.362590 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.364010 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.364278 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.364390 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:39.365546 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:39.365549 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:39.365551 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:39.365564 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:39.365583 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:39.365592 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:39.365652 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:39.365873 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 02:33:39.365931 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 02:33:39.366997 3349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:39.358220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:46629 (local address 127.17.96.195:40671)
0619 02:33:39.358914 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:39.358959 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:39.360430 (+ 1471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:39.361085 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:39.361093 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:39.361186 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:33:39.361738 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:39.361810 (+ 72us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.362731 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.362735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:39.363890 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:39.363903 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.364468 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.364472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:39.364657 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:39.365264 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:33:39.365266 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:33:39.365310 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:33:39.365399 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:33:39.365669 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:33:39.365675 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:33:39.365679 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:33:39.365872 (+ 193us) client_negotiation.cc:240] Negotiation successful
0619 02:33:39.366158 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":119,"threads_started":1}
I20260619 02:33:39.368170 3347 heartbeater.cc:347] Connected to a master server at 127.17.96.254:46629
I20260619 02:33:39.368399 3347 heartbeater.cc:464] Registering TS with master...
I20260619 02:33:39.368966 3347 heartbeater.cc:511] Master 127.17.96.254:46629 requested a full tablet report, sending...
I20260619 02:33:39.370023 2897 ts_manager.cc:194] Registered new tserver with Master: 36ce49f31df940be9a48068c6d28852f (127.17.96.195:41791)
I20260619 02:33:39.370612 2897 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.96.195:40671
I20260619 02:33:39.376394 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:33:39.382344 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:39.378529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34304 (local address 127.17.96.254:46629)
0619 02:33:39.378679 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:39.378683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:39.378812 (+ 129us) server_negotiation.cc:407] Connection header received
0619 02:33:39.378939 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:39.378942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:39.379000 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:39.379073 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:39.379335 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.379932 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.380760 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.381002 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.381124 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:39.381985 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:39.381989 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:39.381991 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:39.382004 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:39.382022 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:39.382029 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:39.382089 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:39.382164 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 02:33:39.382220 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 02:33:39.385421 2897 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:33:39.388664 2897 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:33:39.405377 3360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:39.398662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44800 (local address 127.17.96.194:38841)
0619 02:33:39.399671 (+ 1009us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:39.399690 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:39.400126 (+ 436us) server_negotiation.cc:407] Connection header received
0619 02:33:39.400891 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:39.400896 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:39.401054 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:39.401198 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:39.401522 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.402582 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.404011 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.404633 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.404751 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:39.404981 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:39.405035 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 02:33:39.405174 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":897,"thread_start_us":171,"threads_started":1}
I20260619 02:33:39.405400 3359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:39.398511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:38841 (local address 127.0.0.1:44800)
0619 02:33:39.399882 (+ 1371us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:39.399943 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:39.400115 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:39.401279 (+ 1164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:39.401282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:39.401299 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:39.401403 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:39.401408 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.402710 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.402713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:39.403874 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:39.403885 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.404760 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.404762 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:39.404868 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:39.404905 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 02:33:39.404995 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 02:33:39.405151 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1150,"mutex_wait_us":79,"spinlock_wait_cycles":4992,"thread_start_us":99,"threads_started":1}
I20260619 02:33:39.406736 3361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:39.398790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56836 (local address 127.17.96.195:41791)
0619 02:33:39.399448 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:39.399465 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:39.400064 (+ 599us) server_negotiation.cc:407] Connection header received
0619 02:33:39.400288 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:39.400292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:39.400507 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:39.400673 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:39.400967 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.402406 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.404673 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.405511 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.405648 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:39.405861 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:39.406407 (+ 546us) server_negotiation.cc:299] Negotiation successful
0619 02:33:39.406542 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":140,"threads_started":1}
I20260619 02:33:39.407339 3358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:39.398641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:41791 (local address 127.0.0.1:56836)
0619 02:33:39.399881 (+ 1240us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:39.399943 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:39.400115 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:39.400656 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:39.400663 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:39.400692 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:39.400831 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:39.400836 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.402555 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.402559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:39.404512 (+ 1953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:39.404524 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.406074 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.406079 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:39.406214 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:39.406252 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 02:33:39.407054 (+ 802us) client_negotiation.cc:240] Negotiation successful
0619 02:33:39.407157 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1123,"mutex_wait_us":224,"spinlock_wait_cycles":1408,"thread_start_us":129,"threads_started":1}
I20260619 02:33:39.407984 3363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:39.400272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40220 (local address 127.17.96.193:46129)
0619 02:33:39.400672 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:39.400684 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:39.400705 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:33:39.400778 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:39.400782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:39.400957 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:39.401110 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:39.401387 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.402615 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.405845 (+ 3230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.406656 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.407470 (+ 814us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:39.407671 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:39.407725 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 02:33:39.407830 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":174,"threads_started":1}
I20260619 02:33:39.408144 3362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:39.399791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:46129 (local address 127.0.0.1:40220)
0619 02:33:39.400386 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:39.400403 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:39.400501 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:39.401113 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:39.401116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:39.401136 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:39.401233 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:39.401240 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.402757 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.402760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:39.404512 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:39.404524 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.406982 (+ 2458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.406986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:39.407093 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:39.407118 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 02:33:39.407180 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 02:33:39.407253 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"mutex_wait_us":118,"thread_start_us":99,"threads_started":1}
I20260619 02:33:39.409277 3150 tablet_service.cc:1467] Processing CreateTablet for tablet aeee5dc7966045f2a20f7e61959391f2 (DEFAULT_TABLE table=test-table [id=f3478849c8fc46ddba320acc7952281b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:33:39.410532 3150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeee5dc7966045f2a20f7e61959391f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:39.411394 3281 tablet_service.cc:1467] Processing CreateTablet for tablet aeee5dc7966045f2a20f7e61959391f2 (DEFAULT_TABLE table=test-table [id=f3478849c8fc46ddba320acc7952281b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:33:39.411393 3019 tablet_service.cc:1467] Processing CreateTablet for tablet aeee5dc7966045f2a20f7e61959391f2 (DEFAULT_TABLE table=test-table [id=f3478849c8fc46ddba320acc7952281b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:33:39.412585 3281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeee5dc7966045f2a20f7e61959391f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:39.412585 3019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aeee5dc7966045f2a20f7e61959391f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:39.418304 3364 tablet_bootstrap.cc:492] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a: Bootstrap starting.
I20260619 02:33:39.420696 3365 tablet_bootstrap.cc:492] T aeee5dc7966045f2a20f7e61959391f2 P 36ce49f31df940be9a48068c6d28852f: Bootstrap starting.
I20260619 02:33:39.420949 3364 tablet_bootstrap.cc:654] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:39.421938 3364 log.cc:824] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:39.422186 3366 tablet_bootstrap.cc:492] T aeee5dc7966045f2a20f7e61959391f2 P c1bd7c5629de4b87bceb799b7fbefc57: Bootstrap starting.
I20260619 02:33:39.423280 3365 tablet_bootstrap.cc:654] T aeee5dc7966045f2a20f7e61959391f2 P 36ce49f31df940be9a48068c6d28852f: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:39.424224 3366 tablet_bootstrap.cc:654] T aeee5dc7966045f2a20f7e61959391f2 P c1bd7c5629de4b87bceb799b7fbefc57: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:39.424329 3365 log.cc:824] T aeee5dc7966045f2a20f7e61959391f2 P 36ce49f31df940be9a48068c6d28852f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:39.424953 3366 log.cc:824] T aeee5dc7966045f2a20f7e61959391f2 P c1bd7c5629de4b87bceb799b7fbefc57: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:39.426661 3365 tablet_bootstrap.cc:492] T aeee5dc7966045f2a20f7e61959391f2 P 36ce49f31df940be9a48068c6d28852f: No bootstrap required, opened a new log
I20260619 02:33:39.426823 3366 tablet_bootstrap.cc:492] T aeee5dc7966045f2a20f7e61959391f2 P c1bd7c5629de4b87bceb799b7fbefc57: No bootstrap required, opened a new log
I20260619 02:33:39.426942 3365 ts_tablet_manager.cc:1397] T aeee5dc7966045f2a20f7e61959391f2 P 36ce49f31df940be9a48068c6d28852f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 02:33:39.427040 3366 ts_tablet_manager.cc:1397] T aeee5dc7966045f2a20f7e61959391f2 P c1bd7c5629de4b87bceb799b7fbefc57: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 02:33:39.430934 3366 raft_consensus.cc:348] T aeee5dc7966045f2a20f7e61959391f2 P c1bd7c5629de4b87bceb799b7fbefc57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf6fbed6bc14b47a68c89990f439a2a" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 38841 } } peers { permanent_uuid: "36ce49f31df940be9a48068c6d28852f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 41791 } } peers { permanent_uuid: "c1bd7c5629de4b87bceb799b7fbefc57" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 46129 } }
I20260619 02:33:39.431291 3366 raft_consensus.cc:374] T aeee5dc7966045f2a20f7e61959391f2 P c1bd7c5629de4b87bceb799b7fbefc57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:39.431355 3366 raft_consensus.cc:729] T aeee5dc7966045f2a20f7e61959391f2 P c1bd7c5629de4b87bceb799b7fbefc57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1bd7c5629de4b87bceb799b7fbefc57, State: Initialized, Role: FOLLOWER
I20260619 02:33:39.431316 3365 raft_consensus.cc:348] T aeee5dc7966045f2a20f7e61959391f2 P 36ce49f31df940be9a48068c6d28852f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf6fbed6bc14b47a68c89990f439a2a" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 38841 } } peers { permanent_uuid: "36ce49f31df940be9a48068c6d28852f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 41791 } } peers { permanent_uuid: "c1bd7c5629de4b87bceb799b7fbefc57" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 46129 } }
I20260619 02:33:39.431602 3365 raft_consensus.cc:374] T aeee5dc7966045f2a20f7e61959391f2 P 36ce49f31df940be9a48068c6d28852f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:39.431661 3365 raft_consensus.cc:729] T aeee5dc7966045f2a20f7e61959391f2 P 36ce49f31df940be9a48068c6d28852f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36ce49f31df940be9a48068c6d28852f, State: Initialized, Role: FOLLOWER
I20260619 02:33:39.431936 3366 consensus_queue.cc:260] T aeee5dc7966045f2a20f7e61959391f2 P c1bd7c5629de4b87bceb799b7fbefc57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf6fbed6bc14b47a68c89990f439a2a" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 38841 } } peers { permanent_uuid: "36ce49f31df940be9a48068c6d28852f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 41791 } } peers { permanent_uuid: "c1bd7c5629de4b87bceb799b7fbefc57" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 46129 } }
I20260619 02:33:39.432240 3365 consensus_queue.cc:260] T aeee5dc7966045f2a20f7e61959391f2 P 36ce49f31df940be9a48068c6d28852f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf6fbed6bc14b47a68c89990f439a2a" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 38841 } } peers { permanent_uuid: "36ce49f31df940be9a48068c6d28852f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 41791 } } peers { permanent_uuid: "c1bd7c5629de4b87bceb799b7fbefc57" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 46129 } }
I20260619 02:33:39.432767 3085 heartbeater.cc:503] Master 127.17.96.254:46629 was elected leader, sending a full tablet report...
I20260619 02:33:39.432931 3347 heartbeater.cc:503] Master 127.17.96.254:46629 was elected leader, sending a full tablet report...
I20260619 02:33:39.433175 3366 ts_tablet_manager.cc:1428] T aeee5dc7966045f2a20f7e61959391f2 P c1bd7c5629de4b87bceb799b7fbefc57: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 02:33:39.433319 3365 ts_tablet_manager.cc:1428] T aeee5dc7966045f2a20f7e61959391f2 P 36ce49f31df940be9a48068c6d28852f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260619 02:33:39.433797 3086 tablet.cc:2369] T aeee5dc7966045f2a20f7e61959391f2 P c1bd7c5629de4b87bceb799b7fbefc57: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:33:39.435321 3364 tablet_bootstrap.cc:492] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a: No bootstrap required, opened a new log
I20260619 02:33:39.435611 3364 ts_tablet_manager.cc:1397] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.005s
I20260619 02:33:39.439990 3364 raft_consensus.cc:348] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf6fbed6bc14b47a68c89990f439a2a" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 38841 } } peers { permanent_uuid: "36ce49f31df940be9a48068c6d28852f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 41791 } } peers { permanent_uuid: "c1bd7c5629de4b87bceb799b7fbefc57" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 46129 } }
I20260619 02:33:39.440371 3364 raft_consensus.cc:374] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:39.440446 3364 raft_consensus.cc:729] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdf6fbed6bc14b47a68c89990f439a2a, State: Initialized, Role: FOLLOWER
I20260619 02:33:39.440917 3364 consensus_queue.cc:260] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf6fbed6bc14b47a68c89990f439a2a" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 38841 } } peers { permanent_uuid: "36ce49f31df940be9a48068c6d28852f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 41791 } } peers { permanent_uuid: "c1bd7c5629de4b87bceb799b7fbefc57" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 46129 } }
I20260619 02:33:39.441638 3216 heartbeater.cc:503] Master 127.17.96.254:46629 was elected leader, sending a full tablet report...
I20260619 02:33:39.441938 3364 ts_tablet_manager.cc:1428] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a: Time spent starting tablet: real 0.006s user 0.005s sys 0.003s
I20260619 02:33:39.503356 3372 raft_consensus.cc:482] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:33:39.503567 3372 raft_consensus.cc:504] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf6fbed6bc14b47a68c89990f439a2a" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 38841 } } peers { permanent_uuid: "36ce49f31df940be9a48068c6d28852f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 41791 } } peers { permanent_uuid: "c1bd7c5629de4b87bceb799b7fbefc57" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 46129 } }
I20260619 02:33:39.504761 3372 leader_election.cc:288] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36ce49f31df940be9a48068c6d28852f (127.17.96.195:41791), c1bd7c5629de4b87bceb799b7fbefc57 (127.17.96.193:46129)
I20260619 02:33:39.508953 3363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:39.505278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:35415 (local address 127.17.96.193:46129)
0619 02:33:39.505413 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:39.505417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:39.505556 (+ 139us) server_negotiation.cc:407] Connection header received
0619 02:33:39.505702 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:39.505706 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:39.505818 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:39.505910 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:39.506200 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.506789 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.507862 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.508299 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.508443 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:39.508541 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:39.508714 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 02:33:39.508805 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 02:33:39.509047 3361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:39.505102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:34791 (local address 127.17.96.195:41791)
0619 02:33:39.505235 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:39.505240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:39.505269 (+ 29us) server_negotiation.cc:407] Connection header received
0619 02:33:39.505440 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:39.505444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:39.505508 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:39.505611 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:39.506005 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.506674 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.507901 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.508411 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.508588 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:39.508640 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:39.508844 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 02:33:39.508926 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 02:33:39.509013 3373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:39.505122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:46129 (local address 127.17.96.194:35415)
0619 02:33:39.505453 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:39.505472 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:39.505602 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:39.505923 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:39.505926 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:39.505942 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:39.506073 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:39.506079 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.506889 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.506892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:39.507729 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:39.507739 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.508424 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.508428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:39.508573 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:39.508606 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 02:33:39.508701 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 02:33:39.508825 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":3,"thread_start_us":126,"threads_started":1}
I20260619 02:33:39.509019 3218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:39.504954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:41791 (local address 127.17.96.194:34791)
0619 02:33:39.505113 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:39.505131 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:39.505259 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:39.505659 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:39.505662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:39.505682 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:39.505880 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:39.505888 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.506824 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.506830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:39.507730 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:39.507739 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.508588 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.508594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:39.508714 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:39.508739 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 02:33:39.508822 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 02:33:39.508892 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":40}
I20260619 02:33:39.510046 3301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeee5dc7966045f2a20f7e61959391f2" candidate_uuid: "cdf6fbed6bc14b47a68c89990f439a2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36ce49f31df940be9a48068c6d28852f" is_pre_election: true
I20260619 02:33:39.510046 3039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeee5dc7966045f2a20f7e61959391f2" candidate_uuid: "cdf6fbed6bc14b47a68c89990f439a2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1bd7c5629de4b87bceb799b7fbefc57" is_pre_election: true
I20260619 02:33:39.510396 3039 raft_consensus.cc:2454] T aeee5dc7966045f2a20f7e61959391f2 P c1bd7c5629de4b87bceb799b7fbefc57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdf6fbed6bc14b47a68c89990f439a2a in term 0.
I20260619 02:33:39.510401 3301 raft_consensus.cc:2454] T aeee5dc7966045f2a20f7e61959391f2 P 36ce49f31df940be9a48068c6d28852f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdf6fbed6bc14b47a68c89990f439a2a in term 0.
I20260619 02:33:39.510982 3104 leader_election.cc:302] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 36ce49f31df940be9a48068c6d28852f, c1bd7c5629de4b87bceb799b7fbefc57, cdf6fbed6bc14b47a68c89990f439a2a; no voters:
I20260619 02:33:39.511332 3372 raft_consensus.cc:2781] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:33:39.511420 3372 raft_consensus.cc:482] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:33:39.511471 3372 raft_consensus.cc:3037] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:39.512537 3372 raft_consensus.cc:504] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf6fbed6bc14b47a68c89990f439a2a" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 38841 } } peers { permanent_uuid: "36ce49f31df940be9a48068c6d28852f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 41791 } } peers { permanent_uuid: "c1bd7c5629de4b87bceb799b7fbefc57" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 46129 } }
I20260619 02:33:39.512979 3372 leader_election.cc:288] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a [CANDIDATE]: Term 1 election: Requested vote from peers 36ce49f31df940be9a48068c6d28852f (127.17.96.195:41791), c1bd7c5629de4b87bceb799b7fbefc57 (127.17.96.193:46129)
I20260619 02:33:39.513413 3301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeee5dc7966045f2a20f7e61959391f2" candidate_uuid: "cdf6fbed6bc14b47a68c89990f439a2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36ce49f31df940be9a48068c6d28852f"
I20260619 02:33:39.513446 3039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aeee5dc7966045f2a20f7e61959391f2" candidate_uuid: "cdf6fbed6bc14b47a68c89990f439a2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1bd7c5629de4b87bceb799b7fbefc57"
I20260619 02:33:39.513568 3301 raft_consensus.cc:3037] T aeee5dc7966045f2a20f7e61959391f2 P 36ce49f31df940be9a48068c6d28852f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:39.513568 3039 raft_consensus.cc:3037] T aeee5dc7966045f2a20f7e61959391f2 P c1bd7c5629de4b87bceb799b7fbefc57 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:39.514971 3039 raft_consensus.cc:2454] T aeee5dc7966045f2a20f7e61959391f2 P c1bd7c5629de4b87bceb799b7fbefc57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdf6fbed6bc14b47a68c89990f439a2a in term 1.
I20260619 02:33:39.514971 3301 raft_consensus.cc:2454] T aeee5dc7966045f2a20f7e61959391f2 P 36ce49f31df940be9a48068c6d28852f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdf6fbed6bc14b47a68c89990f439a2a in term 1.
I20260619 02:33:39.515359 3103 leader_election.cc:302] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36ce49f31df940be9a48068c6d28852f, cdf6fbed6bc14b47a68c89990f439a2a; no voters:
I20260619 02:33:39.515571 3372 raft_consensus.cc:2781] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:33:39.515808 3372 raft_consensus.cc:686] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a [term 1 LEADER]: Becoming Leader. State: Replica: cdf6fbed6bc14b47a68c89990f439a2a, State: Running, Role: LEADER
I20260619 02:33:39.516191 3372 consensus_queue.cc:237] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf6fbed6bc14b47a68c89990f439a2a" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 38841 } } peers { permanent_uuid: "36ce49f31df940be9a48068c6d28852f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 41791 } } peers { permanent_uuid: "c1bd7c5629de4b87bceb799b7fbefc57" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 46129 } }
I20260619 02:33:39.519527 2897 catalog_manager.cc:5495] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a reported cstate change: term changed from 0 to 1, leader changed from <none> to cdf6fbed6bc14b47a68c89990f439a2a (127.17.96.194). New cstate: current_term: 1 leader_uuid: "cdf6fbed6bc14b47a68c89990f439a2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf6fbed6bc14b47a68c89990f439a2a" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 38841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36ce49f31df940be9a48068c6d28852f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1bd7c5629de4b87bceb799b7fbefc57" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 46129 } health_report { overall_health: UNKNOWN } } }
I20260619 02:33:39.532897 3360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:39.529042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44812 (local address 127.17.96.194:38841)
0619 02:33:39.529213 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:39.529218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:39.529245 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:33:39.529354 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:39.529362 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:39.529423 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:39.529565 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:33:39.529850 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.530390 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.531320 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:39.531544 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:39.531677 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:39.531988 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:33:39.532457 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:33:39.532602 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:39.532644 (+ 42us) server_negotiation.cc:299] Negotiation successful
0619 02:33:39.532705 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260619 02:33:39.539355 3039 raft_consensus.cc:1261] T aeee5dc7966045f2a20f7e61959391f2 P c1bd7c5629de4b87bceb799b7fbefc57 [term 1 FOLLOWER]: Refusing update from remote peer cdf6fbed6bc14b47a68c89990f439a2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:33:39.539346 3301 raft_consensus.cc:1261] T aeee5dc7966045f2a20f7e61959391f2 P 36ce49f31df940be9a48068c6d28852f [term 1 FOLLOWER]: Refusing update from remote peer cdf6fbed6bc14b47a68c89990f439a2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:33:39.540163 3374 consensus_queue.cc:1035] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1bd7c5629de4b87bceb799b7fbefc57" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 46129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:33:39.540402 3372 consensus_queue.cc:1035] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "36ce49f31df940be9a48068c6d28852f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:33:39.548310 3377 mvcc.cc:205] Tried to move back new op lower bound from 7298401974424190976 to 7298401974341185536. Current Snapshot: MvccSnapshot[applied={T|T < 7298401974424190976}]
I20260619 02:33:39.548600 3379 mvcc.cc:205] Tried to move back new op lower bound from 7298401974424190976 to 7298401974341185536. Current Snapshot: MvccSnapshot[applied={T|T < 7298401974424190976}]
I20260619 02:33:39.558087 2896 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:33:39.558308 2896 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:33:39.561002 2896 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91c074ee4b8a47d9be5d473ee6cd88bd: Sending DeleteTablet for 3 replicas of tablet aeee5dc7966045f2a20f7e61959391f2
I20260619 02:33:39.562042 3281 tablet_service.cc:1514] Processing DeleteTablet for tablet aeee5dc7966045f2a20f7e61959391f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:33:39 UTC) from {username='slave'} at 127.0.0.1:56836
I20260619 02:33:39.562137 3019 tablet_service.cc:1514] Processing DeleteTablet for tablet aeee5dc7966045f2a20f7e61959391f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:33:39 UTC) from {username='slave'} at 127.0.0.1:40220
I20260619 02:33:39.562269 3150 tablet_service.cc:1514] Processing DeleteTablet for tablet aeee5dc7966045f2a20f7e61959391f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:33:39 UTC) from {username='slave'} at 127.0.0.1:44800
I20260619 02:33:39.562697 3388 tablet_replica.cc:330] T aeee5dc7966045f2a20f7e61959391f2 P c1bd7c5629de4b87bceb799b7fbefc57: stopping tablet replica
I20260619 02:33:39.562711 3387 tablet_replica.cc:330] T aeee5dc7966045f2a20f7e61959391f2 P 36ce49f31df940be9a48068c6d28852f: stopping tablet replica
I20260619 02:33:39.563093 3387 raft_consensus.cc:2229] T aeee5dc7966045f2a20f7e61959391f2 P 36ce49f31df940be9a48068c6d28852f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:33:39.563243 3389 tablet_replica.cc:330] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a: stopping tablet replica
I20260619 02:33:39.563484 3387 raft_consensus.cc:2258] T aeee5dc7966045f2a20f7e61959391f2 P 36ce49f31df940be9a48068c6d28852f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:33:39.563517 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 2957
I20260619 02:33:39.565418 3389 raft_consensus.cc:2229] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:33:39.565862 3389 raft_consensus.cc:2258] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:33:39.567364 3389 ts_tablet_manager.cc:1905] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:33:39.567365 3387 ts_tablet_manager.cc:1905] T aeee5dc7966045f2a20f7e61959391f2 P 36ce49f31df940be9a48068c6d28852f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:33:39.570384 3389 ts_tablet_manager.cc:1918] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:33:39.570504 3389 log.cc:1196] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/aeee5dc7966045f2a20f7e61959391f2
I20260619 02:33:39.570829 3389 ts_tablet_manager.cc:1939] T aeee5dc7966045f2a20f7e61959391f2 P cdf6fbed6bc14b47a68c89990f439a2a: Deleting consensus metadata
I20260619 02:33:39.571372 3387 ts_tablet_manager.cc:1918] T aeee5dc7966045f2a20f7e61959391f2 P 36ce49f31df940be9a48068c6d28852f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:33:39.571549 3387 log.cc:1196] T aeee5dc7966045f2a20f7e61959391f2 P 36ce49f31df940be9a48068c6d28852f: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/aeee5dc7966045f2a20f7e61959391f2
I20260619 02:33:39.571960 3387 ts_tablet_manager.cc:1939] T aeee5dc7966045f2a20f7e61959391f2 P 36ce49f31df940be9a48068c6d28852f: Deleting consensus metadata
I20260619 02:33:39.572238 2882 catalog_manager.cc:4841] TS cdf6fbed6bc14b47a68c89990f439a2a (127.17.96.194:38841): tablet aeee5dc7966045f2a20f7e61959391f2 (table test-table [id=f3478849c8fc46ddba320acc7952281b]) successfully deleted
W20260619 02:33:39.572742 2882 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.96.193:46129 (error 108)
I20260619 02:33:39.573722 2881 catalog_manager.cc:4841] TS 36ce49f31df940be9a48068c6d28852f (127.17.96.195:41791): tablet aeee5dc7966045f2a20f7e61959391f2 (table test-table [id=f3478849c8fc46ddba320acc7952281b]) successfully deleted
I20260619 02:33:39.573978 3362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:39.573585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:46129 (local address 127.0.0.1:40228)
0619 02:33:39.573807 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:39.573883 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.193:46129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260619 02:33:39.574148 2882 catalog_manager.cc:4584] TS c1bd7c5629de4b87bceb799b7fbefc57 (127.17.96.193:46129): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet aeee5dc7966045f2a20f7e61959391f2: Network error: Client connection negotiation failed: client connection to 127.17.96.193:46129: connect: Connection refused (error 111)
I20260619 02:33:39.574412 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 3088
I20260619 02:33:39.580442 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 3219
I20260619 02:33:39.586570 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 2866
2026-06-19T02:33:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1148 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:33:39 dist-test-slave-4qw6 krb5kdc[3393](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:33:39 dist-test-slave-4qw6 krb5kdc[3393](info): set up 2 sockets
Jun 19 02:33:39 dist-test-slave-4qw6 krb5kdc[3393](info): commencing operation
krb5kdc: starting...
W20260619 02:33:40.637885 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:33:40 dist-test-slave-4qw6 krb5kdc[3393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:33:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:33:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:33:40.769135 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:45653
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:45653
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:33:40.888896 3406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:40.889249 3406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:40.889297 3406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:40.893828 3406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:33:40.893935 3406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:40.893956 3406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:40.893975 3406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:33:40.893991 3406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:33:40.899750 3406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:45653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:45653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:40.900928 3406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:40.901988 3406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:40.911212 3411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:40.911203 3412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:40.911203 3414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:40.911739 3406 server_base.cc:1034] running on GCE node
I20260619 02:33:40.912357 3406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:40.913631 3406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:40.914846 3406 hybrid_clock.cc:648] HybridClock initialized: now 1781836420914838 us; error 36 us; skew 500 ppm
Jun 19 02:33:40 dist-test-slave-4qw6 krb5kdc[3393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836420, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:40.917914 3406 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:33:40.918970 3406 webserver.cc:460] Webserver started at http://127.17.96.254:46831/ using document root <none> and password file <none>
I20260619 02:33:40.919589 3406 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:40.919687 3406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:40.919919 3406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:40.922204 3406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "494925c62c7e4812a9de80b16dfc3d4c"
format_stamp: "Formatted at 2026-06-19 02:33:40 on dist-test-slave-4qw6"
server_key: "19600225877ebd240ca5feae29ddca8d"
server_key_iv: "b153c2c53479c885974f99ed576fd6ee"
server_key_version: "encryptionkey@0"
I20260619 02:33:40.922808 3406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "494925c62c7e4812a9de80b16dfc3d4c"
format_stamp: "Formatted at 2026-06-19 02:33:40 on dist-test-slave-4qw6"
server_key: "19600225877ebd240ca5feae29ddca8d"
server_key_iv: "b153c2c53479c885974f99ed576fd6ee"
server_key_version: "encryptionkey@0"
I20260619 02:33:40.926941 3406 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 02:33:40.929610 3421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:40.931041 3406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:33:40.931224 3406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "494925c62c7e4812a9de80b16dfc3d4c"
format_stamp: "Formatted at 2026-06-19 02:33:40 on dist-test-slave-4qw6"
server_key: "19600225877ebd240ca5feae29ddca8d"
server_key_iv: "b153c2c53479c885974f99ed576fd6ee"
server_key_version: "encryptionkey@0"
I20260619 02:33:40.931365 3406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:40.948328 3406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:40.951798 3406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:40.952062 3406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:40.960079 3406 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:45653
I20260619 02:33:40.960094 3473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:45653 every 8 connection(s)
I20260619 02:33:40.961198 3406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:33:40.964367 3474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:40.966544 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 3406
I20260619 02:33:40.966711 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:33:40.967044 17795 external_mini_cluster.cc:1428] Setting key 334a280fad54970e268fd48403f7e0a7
I20260619 02:33:40.971866 3474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c: Bootstrap starting.
I20260619 02:33:40.974525 3474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:40.975368 3474 log.cc:824] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c: Log is configured to *not* fsync() on all Append() calls
Jun 19 02:33:40 dist-test-slave-4qw6 krb5kdc[3393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:33:40.978196 3474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c: No bootstrap required, opened a new log
I20260619 02:33:40.982640 3474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "494925c62c7e4812a9de80b16dfc3d4c" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45653 } }
I20260619 02:33:40.982900 3474 raft_consensus.cc:374] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:40.982973 3474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 494925c62c7e4812a9de80b16dfc3d4c, State: Initialized, Role: FOLLOWER
I20260619 02:33:40.983222 3477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:40.968709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60960 (local address 127.17.96.254:45653)
0619 02:33:40.969255 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:40.969271 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:40.969311 (+ 40us) server_negotiation.cc:407] Connection header received
0619 02:33:40.970128 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:40.970149 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:40.970531 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:40.970903 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:40.971835 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:40.972945 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:40.973922 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:40.974429 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:40.974623 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:40.977816 (+ 3193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:40.977825 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:40.977845 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:33:40.977880 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:40.979811 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:40.980486 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:40.980493 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:40.980498 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:40.980626 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:40.980926 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:40.980929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:40.980930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:40.981364 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:40.981528 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:40.981925 (+ 397us) server_negotiation.cc:299] Negotiation successful
0619 02:33:40.982276 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":174,"threads_started":1}
I20260619 02:33:40.983510 3474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "494925c62c7e4812a9de80b16dfc3d4c" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45653 } }
I20260619 02:33:40.983685 3474 raft_consensus.cc:388] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:33:40.983772 3474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:33:40.983901 3474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:40.984938 3474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "494925c62c7e4812a9de80b16dfc3d4c" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45653 } }
I20260619 02:33:40.985323 3474 leader_election.cc:302] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 494925c62c7e4812a9de80b16dfc3d4c; no voters:
I20260619 02:33:40.985721 3474 leader_election.cc:288] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:33:40.985993 3479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:33:40.986183 3479 raft_consensus.cc:686] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c [term 1 LEADER]: Becoming Leader. State: Replica: 494925c62c7e4812a9de80b16dfc3d4c, State: Running, Role: LEADER
I20260619 02:33:40.986495 3479 consensus_queue.cc:237] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "494925c62c7e4812a9de80b16dfc3d4c" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45653 } }
I20260619 02:33:40.987159 3474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:33:40.988286 3481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 494925c62c7e4812a9de80b16dfc3d4c. Latest consensus state: current_term: 1 leader_uuid: "494925c62c7e4812a9de80b16dfc3d4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "494925c62c7e4812a9de80b16dfc3d4c" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45653 } } }
I20260619 02:33:40.988428 3481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:40.988793 3480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "494925c62c7e4812a9de80b16dfc3d4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "494925c62c7e4812a9de80b16dfc3d4c" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 45653 } } }
I20260619 02:33:40.988904 3480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:40.989169 3488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:33:40.992708 3488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:33:40.999610 3488 catalog_manager.cc:1347] Generated new cluster ID: 89b3b42397d740808ee623e10d6994d0
I20260619 02:33:40.999722 3488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:33:41.032621 3488 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:33:41.033723 3488 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:33:41.039099 3488 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c: Generated new TSK 0
I20260619 02:33:41.039938 3488 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.96.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
I20260619 02:33:41.073381 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45653
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:33:41.198448 3500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:41.198766 3500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:41.198822 3500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:41.203202 3500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:41.203287 3500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:41.203377 3500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:33:41.209276 3500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:41.210592 3500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:41.211722 3500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:41.222237 3506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:41.222234 3505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:41.222230 3508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:41.222293 3500 server_base.cc:1034] running on GCE node
I20260619 02:33:41.222988 3500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:41.223694 3500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:41.224886 3500 hybrid_clock.cc:648] HybridClock initialized: now 1781836421224870 us; error 36 us; skew 500 ppm
Jun 19 02:33:41 dist-test-slave-4qw6 krb5kdc[3393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836421, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:41.228046 3500 init.cc:377] Logged in from keytab as kudu/127.17.96.193@KRBTEST.COM (short username kudu)
I20260619 02:33:41.229113 3500 webserver.cc:460] Webserver started at http://127.17.96.193:46111/ using document root <none> and password file <none>
I20260619 02:33:41.229832 3500 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:41.229933 3500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:41.230185 3500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:41.232524 3500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "db5f3215df92477f99cd452313a01ae0"
format_stamp: "Formatted at 2026-06-19 02:33:41 on dist-test-slave-4qw6"
server_key: "fcb4cb4c98a77cc2e7a314dd3fbe0b2b"
server_key_iv: "4cb202173f91fec185e1cbf1c3a155a1"
server_key_version: "encryptionkey@0"
I20260619 02:33:41.233186 3500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "db5f3215df92477f99cd452313a01ae0"
format_stamp: "Formatted at 2026-06-19 02:33:41 on dist-test-slave-4qw6"
server_key: "fcb4cb4c98a77cc2e7a314dd3fbe0b2b"
server_key_iv: "4cb202173f91fec185e1cbf1c3a155a1"
server_key_version: "encryptionkey@0"
I20260619 02:33:41.237892 3500 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.005s
I20260619 02:33:41.240743 3515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:41.242214 3500 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:33:41.242395 3500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "db5f3215df92477f99cd452313a01ae0"
format_stamp: "Formatted at 2026-06-19 02:33:41 on dist-test-slave-4qw6"
server_key: "fcb4cb4c98a77cc2e7a314dd3fbe0b2b"
server_key_iv: "4cb202173f91fec185e1cbf1c3a155a1"
server_key_version: "encryptionkey@0"
I20260619 02:33:41.242537 3500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:41.262181 3500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:41.265429 3500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:41.265671 3500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:41.266351 3500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:33:41.267298 3500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:33:41.267374 3500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:41.267452 3500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:33:41.267503 3500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:41.277448 3500 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:33837
I20260619 02:33:41.277473 3628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:33837 every 8 connection(s)
I20260619 02:33:41.278564 3500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:33:41.282516 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 3500
I20260619 02:33:41.282649 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:33:41.283037 17795 external_mini_cluster.cc:1428] Setting key d69ee166b28d56e8cd893ef715942101
Jun 19 02:33:41 dist-test-slave-4qw6 krb5kdc[3393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836421, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.96.194@KRBTEST.COM; defaulting to no policy
I20260619 02:33:41.295644 3477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:41.280716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:57623 (local address 127.17.96.254:45653)
0619 02:33:41.280881 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:41.280886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:41.281919 (+ 1033us) server_negotiation.cc:407] Connection header received
0619 02:33:41.283734 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:41.283738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:41.283807 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:41.283894 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:41.285262 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.285900 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.287157 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.287620 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.287750 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:41.290956 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:41.290963 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:41.290969 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:33:41.291008 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:41.292860 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:41.293485 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:41.293491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:41.293495 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:41.293603 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:41.294174 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:41.294177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:41.294178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:41.294468 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:41.294604 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:41.295209 (+ 605us) server_negotiation.cc:299] Negotiation successful
0619 02:33:41.295427 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.194@KRBTEST.COM" created.
I20260619 02:33:41.296618 3631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:41.281163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45653 (local address 127.17.96.193:57623)
0619 02:33:41.281664 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:41.281749 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:41.283477 (+ 1728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:41.284132 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:41.284140 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:41.284539 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:33:41.285088 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:41.285104 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.286039 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.286043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:41.287008 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:41.287020 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.287795 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.287798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:41.287976 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:41.288788 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:33:41.288807 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:33:41.290719 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:33:41.293025 (+ 2306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:33:41.293031 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:33:41.293049 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:33:41.293348 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:33:41.293884 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:33:41.293887 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:33:41.293888 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:33:41.294065 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:33:41.294604 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:33:41.294614 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:33:41.294932 (+ 318us) client_negotiation.cc:769] Sending connection context
0619 02:33:41.295150 (+ 218us) client_negotiation.cc:240] Negotiation successful
0619 02:33:41.295540 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":117,"threads_started":1}
I20260619 02:33:41.298141 3629 heartbeater.cc:347] Connected to a master server at 127.17.96.254:45653
I20260619 02:33:41.298439 3629 heartbeater.cc:464] Registering TS with master...
I20260619 02:33:41.299000 3629 heartbeater.cc:511] Master 127.17.96.254:45653 requested a full tablet report, sending...
I20260619 02:33:41.300812 3438 ts_manager.cc:194] Registered new tserver with Master: db5f3215df92477f99cd452313a01ae0 (127.17.96.193:33837)
I20260619 02:33:41.302724 3438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:57623
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
I20260619 02:33:41.313212 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45653
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:33:41.431478 3634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:41.431795 3634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:41.431843 3634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:41.436220 3634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:41.436307 3634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:41.436403 3634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:33:41.442422 3634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:41.443663 3634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:41.444794 3634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:41.456012 3640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:41.456036 3642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:41.456146 3634 server_base.cc:1034] running on GCE node
W20260619 02:33:41.456012 3639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:41.456846 3634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:41.457625 3634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:41.458827 3634 hybrid_clock.cc:648] HybridClock initialized: now 1781836421458787 us; error 53 us; skew 500 ppm
Jun 19 02:33:41 dist-test-slave-4qw6 krb5kdc[3393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836421, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:41.461992 3634 init.cc:377] Logged in from keytab as kudu/127.17.96.194@KRBTEST.COM (short username kudu)
I20260619 02:33:41.463135 3634 webserver.cc:460] Webserver started at http://127.17.96.194:44967/ using document root <none> and password file <none>
I20260619 02:33:41.463762 3634 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:41.463871 3634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:41.464123 3634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:41.466470 3634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "1d38053e7d3f421dbb71e522b0ac13ee"
format_stamp: "Formatted at 2026-06-19 02:33:41 on dist-test-slave-4qw6"
server_key: "48e2e231ae0179056ad186876f70906b"
server_key_iv: "2fb10d5565377bb7e0d21dabbb79b610"
server_key_version: "encryptionkey@0"
I20260619 02:33:41.467120 3634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "1d38053e7d3f421dbb71e522b0ac13ee"
format_stamp: "Formatted at 2026-06-19 02:33:41 on dist-test-slave-4qw6"
server_key: "48e2e231ae0179056ad186876f70906b"
server_key_iv: "2fb10d5565377bb7e0d21dabbb79b610"
server_key_version: "encryptionkey@0"
I20260619 02:33:41.471328 3634 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 02:33:41.474124 3649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:41.475530 3634 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 02:33:41.475692 3634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "1d38053e7d3f421dbb71e522b0ac13ee"
format_stamp: "Formatted at 2026-06-19 02:33:41 on dist-test-slave-4qw6"
server_key: "48e2e231ae0179056ad186876f70906b"
server_key_iv: "2fb10d5565377bb7e0d21dabbb79b610"
server_key_version: "encryptionkey@0"
I20260619 02:33:41.475834 3634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:41.490278 3634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:41.494021 3634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:41.494249 3634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:41.494894 3634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:33:41.495834 3634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:33:41.495908 3634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:41.495985 3634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:33:41.496037 3634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:41.506026 3634 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:43819
I20260619 02:33:41.506045 3762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:43819 every 8 connection(s)
I20260619 02:33:41.507061 3634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:33:41.510854 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 3634
I20260619 02:33:41.510977 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:33:41.511289 17795 external_mini_cluster.cc:1428] Setting key 62c8c81b842b532f40fbacad455aba41
Jun 19 02:33:41 dist-test-slave-4qw6 krb5kdc[3393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836421, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.96.195@KRBTEST.COM; defaulting to no policy
I20260619 02:33:41.523391 3477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:41.509161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:34655 (local address 127.17.96.254:45653)
0619 02:33:41.509364 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:41.509369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:41.510345 (+ 976us) server_negotiation.cc:407] Connection header received
0619 02:33:41.511978 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:41.511982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:41.512037 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:41.512147 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:41.513619 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.514299 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.515580 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.515835 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.515962 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:41.519173 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:41.519178 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:41.519180 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:33:41.519210 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:41.520891 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:41.521581 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:41.521585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:41.521587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:41.521674 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:41.522086 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:41.522091 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:41.522092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:41.522337 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:41.522466 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:41.523056 (+ 590us) server_negotiation.cc:299] Negotiation successful
0619 02:33:41.523206 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.195@KRBTEST.COM" created.
I20260619 02:33:41.524390 3765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:41.509600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45653 (local address 127.17.96.194:34655)
0619 02:33:41.510166 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:41.510216 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:41.511709 (+ 1493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:41.512356 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:41.512365 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:41.512904 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:33:41.513453 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:41.513468 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.514453 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.514464 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:41.515421 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:41.515432 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.516018 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.516022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:41.516219 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:41.516966 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:33:41.516995 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:33:41.518947 (+ 1952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:33:41.521042 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:33:41.521050 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:33:41.521070 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:33:41.521446 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:33:41.521800 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:33:41.521804 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:33:41.521806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:33:41.521977 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:33:41.522462 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:33:41.522470 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:33:41.522779 (+ 309us) client_negotiation.cc:769] Sending connection context
0619 02:33:41.522994 (+ 215us) client_negotiation.cc:240] Negotiation successful
0619 02:33:41.523376 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":193,"threads_started":1}
I20260619 02:33:41.525817 3763 heartbeater.cc:347] Connected to a master server at 127.17.96.254:45653
I20260619 02:33:41.526154 3763 heartbeater.cc:464] Registering TS with master...
I20260619 02:33:41.526712 3763 heartbeater.cc:511] Master 127.17.96.254:45653 requested a full tablet report, sending...
I20260619 02:33:41.527966 3438 ts_manager.cc:194] Registered new tserver with Master: 1d38053e7d3f421dbb71e522b0ac13ee (127.17.96.194:43819)
I20260619 02:33:41.528652 3438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:34655
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
I20260619 02:33:41.541431 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45653
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:33:41.660995 3768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:41.661314 3768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:41.661361 3768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:41.665874 3768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:41.665959 3768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:41.666023 3768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:33:41.671859 3768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:45653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:41.673098 3768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:41.674263 3768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:41.684796 3776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:41.684839 3768 server_base.cc:1034] running on GCE node
W20260619 02:33:41.684793 3774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:41.684793 3773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:41.685567 3768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:41.686369 3768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:41.687536 3768 hybrid_clock.cc:648] HybridClock initialized: now 1781836421687500 us; error 52 us; skew 500 ppm
Jun 19 02:33:41 dist-test-slave-4qw6 krb5kdc[3393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836421, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:41.690734 3768 init.cc:377] Logged in from keytab as kudu/127.17.96.195@KRBTEST.COM (short username kudu)
I20260619 02:33:41.691715 3768 webserver.cc:460] Webserver started at http://127.17.96.195:33073/ using document root <none> and password file <none>
I20260619 02:33:41.692278 3768 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:41.692349 3768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:41.692531 3768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:41.694756 3768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "7f8926e1fc61423f9af7baa19c332c36"
format_stamp: "Formatted at 2026-06-19 02:33:41 on dist-test-slave-4qw6"
server_key: "a78f402cceb9c4f2fd9cc1f5df8df12c"
server_key_iv: "b7c4a6f4b0b811f7f75f2ae868929a7c"
server_key_version: "encryptionkey@0"
I20260619 02:33:41.695302 3768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "7f8926e1fc61423f9af7baa19c332c36"
format_stamp: "Formatted at 2026-06-19 02:33:41 on dist-test-slave-4qw6"
server_key: "a78f402cceb9c4f2fd9cc1f5df8df12c"
server_key_iv: "b7c4a6f4b0b811f7f75f2ae868929a7c"
server_key_version: "encryptionkey@0"
I20260619 02:33:41.699393 3768 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:33:41.702219 3783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:41.703765 3768 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 02:33:41.703971 3768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "7f8926e1fc61423f9af7baa19c332c36"
format_stamp: "Formatted at 2026-06-19 02:33:41 on dist-test-slave-4qw6"
server_key: "a78f402cceb9c4f2fd9cc1f5df8df12c"
server_key_iv: "b7c4a6f4b0b811f7f75f2ae868929a7c"
server_key_version: "encryptionkey@0"
I20260619 02:33:41.704119 3768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:41.727947 3768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:41.731585 3768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:41.731844 3768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:41.732499 3768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:33:41.733445 3768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:33:41.733521 3768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:41.733598 3768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:33:41.733649 3768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:41.743884 3768 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:38797
I20260619 02:33:41.743899 3896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:38797 every 8 connection(s)
I20260619 02:33:41.744962 3768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:33:41.749402 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 3768
I20260619 02:33:41.749564 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:33:41.749961 17795 external_mini_cluster.cc:1428] Setting key 8da56a06e493eed8d7b6ebdff5a7db06
Jun 19 02:33:41 dist-test-slave-4qw6 krb5kdc[3393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836421, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:33:41.760601 3477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:41.747191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:37621 (local address 127.17.96.254:45653)
0619 02:33:41.747330 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:41.747334 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:41.748290 (+ 956us) server_negotiation.cc:407] Connection header received
0619 02:33:41.750021 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:41.750028 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:41.750110 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:41.750189 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:41.751867 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.752530 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.753808 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.754128 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.754249 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:41.757052 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:41.757056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:41.757058 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:33:41.757088 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:41.758589 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:41.759156 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:41.759159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:41.759160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:41.759245 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:41.759555 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:41.759558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:41.759559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:41.759758 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:41.759837 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:41.760340 (+ 503us) server_negotiation.cc:299] Negotiation successful
0619 02:33:41.760452 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:33:41.761484 3899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:41.747606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:45653 (local address 127.17.96.195:37621)
0619 02:33:41.748110 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:41.748158 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:41.749702 (+ 1544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:41.750431 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:41.750439 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:41.750985 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:33:41.751713 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:41.751731 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.752667 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.752670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:41.753615 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:41.753626 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.754300 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.754304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:41.754506 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:41.755199 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:33:41.755218 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:33:41.756832 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:33:41.758710 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:33:41.758716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:33:41.758730 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:33:41.759047 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:33:41.759331 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:33:41.759333 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:33:41.759335 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:33:41.759474 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:33:41.759861 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:33:41.759867 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:33:41.760126 (+ 259us) client_negotiation.cc:769] Sending connection context
0619 02:33:41.760292 (+ 166us) client_negotiation.cc:240] Negotiation successful
0619 02:33:41.760589 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":135,"threads_started":1}
I20260619 02:33:41.762748 3897 heartbeater.cc:347] Connected to a master server at 127.17.96.254:45653
I20260619 02:33:41.763017 3897 heartbeater.cc:464] Registering TS with master...
I20260619 02:33:41.763624 3897 heartbeater.cc:511] Master 127.17.96.254:45653 requested a full tablet report, sending...
I20260619 02:33:41.764750 3438 ts_manager.cc:194] Registered new tserver with Master: 7f8926e1fc61423f9af7baa19c332c36 (127.17.96.195:38797)
I20260619 02:33:41.765314 3438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:37621
I20260619 02:33:41.776448 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:33:41.787320 3477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:41.778593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60970 (local address 127.17.96.254:45653)
0619 02:33:41.778780 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:41.778785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:41.778910 (+ 125us) server_negotiation.cc:407] Connection header received
0619 02:33:41.779079 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:41.779082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:41.779136 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:41.779224 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:41.780027 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.780589 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.781467 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.781693 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.781833 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:41.783894 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:41.783899 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:41.783901 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:33:41.783938 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:41.785170 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:41.785742 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:41.785772 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:41.785774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:41.785915 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:41.786313 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:41.786316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:41.786318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:41.786572 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:41.786748 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:41.787047 (+ 299us) server_negotiation.cc:299] Negotiation successful
0619 02:33:41.787160 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260619 02:33:41.790684 3438 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60970:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:33:41.793965 3438 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:33:41.810261 3910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:41.803720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41696 (local address 127.17.96.195:38797)
0619 02:33:41.804791 (+ 1071us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:41.804804 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:41.804830 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:33:41.804933 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:41.804939 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:41.805160 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:41.805329 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:41.805654 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.806947 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.808470 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.809373 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.809488 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:41.809631 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:41.810041 (+ 410us) server_negotiation.cc:299] Negotiation successful
0619 02:33:41.810131 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":961,"thread_start_us":165,"threads_started":1}
I20260619 02:33:41.810240 3911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:41.804289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46558 (local address 127.17.96.193:33837)
0619 02:33:41.805005 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:41.805010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:41.805027 (+ 17us) server_negotiation.cc:407] Connection header received
0619 02:33:41.805126 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:41.805130 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:41.805310 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:41.805461 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:41.805877 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.806739 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.808309 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.809218 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.809328 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:41.809503 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:41.809967 (+ 464us) server_negotiation.cc:299] Negotiation successful
0619 02:33:41.810082 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":103,"threads_started":1}
I20260619 02:33:41.810245 3908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:41.803568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:33837 (local address 127.0.0.1:46558)
0619 02:33:41.804449 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:41.804464 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:41.804546 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:41.805612 (+ 1066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:41.805615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:41.805630 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:41.805731 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:41.805735 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.806853 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.806860 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:41.808175 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:41.808187 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.809573 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.809577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:41.809705 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:41.809772 (+ 67us) client_negotiation.cc:769] Sending connection context
0619 02:33:41.809906 (+ 134us) client_negotiation.cc:240] Negotiation successful
0619 02:33:41.810059 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":806,"mutex_wait_us":56,"thread_start_us":117,"threads_started":1}
I20260619 02:33:41.810932 3909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:41.803549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:38797 (local address 127.0.0.1:41696)
0619 02:33:41.804006 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:41.804058 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:41.804192 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:41.805317 (+ 1125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:41.805323 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:41.805362 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:41.805516 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:41.805525 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.807108 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.807111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:41.808311 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:41.808320 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.809695 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.809701 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:41.809830 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:41.809859 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 02:33:41.809956 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 02:33:41.810818 (+ 862us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":799,"thread_start_us":114,"threads_started":1}
I20260619 02:33:41.811198 3912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:41.804329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:43819 (local address 127.0.0.1:40720)
0619 02:33:41.804681 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:41.804696 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:41.804795 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:41.806104 (+ 1309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:41.806107 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:41.806126 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:41.806206 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:41.806212 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.807755 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.807758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:41.809201 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:41.809212 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.810806 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.810810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:41.810898 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:41.810932 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 02:33:41.810999 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 02:33:41.811072 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":251,"thread_start_us":94,"threads_started":1}
I20260619 02:33:41.811918 3913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:41.804511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40720 (local address 127.17.96.194:43819)
0619 02:33:41.805640 (+ 1129us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:41.805651 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:41.805672 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:33:41.805743 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:41.805748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:41.805955 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:41.806105 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:41.806333 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.807428 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.809316 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.810179 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.810291 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:41.811601 (+ 1310us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:41.811648 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 02:33:41.811728 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1020,"thread_start_us":118,"threads_started":1}
I20260619 02:33:41.814100 3563 tablet_service.cc:1467] Processing CreateTablet for tablet faee3b36fdbf4f4cbba46830b3fd389f (DEFAULT_TABLE table=test-table [id=7f6079c2ba2b465a95add29437e164a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:33:41.814106 3831 tablet_service.cc:1467] Processing CreateTablet for tablet faee3b36fdbf4f4cbba46830b3fd389f (DEFAULT_TABLE table=test-table [id=7f6079c2ba2b465a95add29437e164a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:33:41.814704 3697 tablet_service.cc:1467] Processing CreateTablet for tablet faee3b36fdbf4f4cbba46830b3fd389f (DEFAULT_TABLE table=test-table [id=7f6079c2ba2b465a95add29437e164a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:33:41.815194 3563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faee3b36fdbf4f4cbba46830b3fd389f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:41.815199 3831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faee3b36fdbf4f4cbba46830b3fd389f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:41.815668 3697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faee3b36fdbf4f4cbba46830b3fd389f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:41.822965 3914 tablet_bootstrap.cc:492] T faee3b36fdbf4f4cbba46830b3fd389f P db5f3215df92477f99cd452313a01ae0: Bootstrap starting.
I20260619 02:33:41.823467 3915 tablet_bootstrap.cc:492] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36: Bootstrap starting.
I20260619 02:33:41.825567 3915 tablet_bootstrap.cc:654] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:41.826243 3916 tablet_bootstrap.cc:492] T faee3b36fdbf4f4cbba46830b3fd389f P 1d38053e7d3f421dbb71e522b0ac13ee: Bootstrap starting.
I20260619 02:33:41.827081 3915 log.cc:824] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:41.827939 3914 tablet_bootstrap.cc:654] T faee3b36fdbf4f4cbba46830b3fd389f P db5f3215df92477f99cd452313a01ae0: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:41.828311 3916 tablet_bootstrap.cc:654] T faee3b36fdbf4f4cbba46830b3fd389f P 1d38053e7d3f421dbb71e522b0ac13ee: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:41.828882 3914 log.cc:824] T faee3b36fdbf4f4cbba46830b3fd389f P db5f3215df92477f99cd452313a01ae0: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:41.829622 3916 log.cc:824] T faee3b36fdbf4f4cbba46830b3fd389f P 1d38053e7d3f421dbb71e522b0ac13ee: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:41.829913 3915 tablet_bootstrap.cc:492] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36: No bootstrap required, opened a new log
I20260619 02:33:41.830140 3915 ts_tablet_manager.cc:1397] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260619 02:33:41.830819 3914 tablet_bootstrap.cc:492] T faee3b36fdbf4f4cbba46830b3fd389f P db5f3215df92477f99cd452313a01ae0: No bootstrap required, opened a new log
I20260619 02:33:41.831032 3914 ts_tablet_manager.cc:1397] T faee3b36fdbf4f4cbba46830b3fd389f P db5f3215df92477f99cd452313a01ae0: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260619 02:33:41.831574 3916 tablet_bootstrap.cc:492] T faee3b36fdbf4f4cbba46830b3fd389f P 1d38053e7d3f421dbb71e522b0ac13ee: No bootstrap required, opened a new log
I20260619 02:33:41.831743 3916 ts_tablet_manager.cc:1397] T faee3b36fdbf4f4cbba46830b3fd389f P 1d38053e7d3f421dbb71e522b0ac13ee: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 02:33:41.834324 3915 raft_consensus.cc:348] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f8926e1fc61423f9af7baa19c332c36" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38797 } } peers { permanent_uuid: "db5f3215df92477f99cd452313a01ae0" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 33837 } } peers { permanent_uuid: "1d38053e7d3f421dbb71e522b0ac13ee" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43819 } }
I20260619 02:33:41.834355 3914 raft_consensus.cc:348] T faee3b36fdbf4f4cbba46830b3fd389f P db5f3215df92477f99cd452313a01ae0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f8926e1fc61423f9af7baa19c332c36" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38797 } } peers { permanent_uuid: "db5f3215df92477f99cd452313a01ae0" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 33837 } } peers { permanent_uuid: "1d38053e7d3f421dbb71e522b0ac13ee" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43819 } }
I20260619 02:33:41.834642 3915 raft_consensus.cc:374] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:41.834674 3914 raft_consensus.cc:374] T faee3b36fdbf4f4cbba46830b3fd389f P db5f3215df92477f99cd452313a01ae0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:41.834703 3915 raft_consensus.cc:729] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f8926e1fc61423f9af7baa19c332c36, State: Initialized, Role: FOLLOWER
I20260619 02:33:41.834724 3914 raft_consensus.cc:729] T faee3b36fdbf4f4cbba46830b3fd389f P db5f3215df92477f99cd452313a01ae0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db5f3215df92477f99cd452313a01ae0, State: Initialized, Role: FOLLOWER
I20260619 02:33:41.835218 3915 consensus_queue.cc:260] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f8926e1fc61423f9af7baa19c332c36" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38797 } } peers { permanent_uuid: "db5f3215df92477f99cd452313a01ae0" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 33837 } } peers { permanent_uuid: "1d38053e7d3f421dbb71e522b0ac13ee" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43819 } }
I20260619 02:33:41.835218 3914 consensus_queue.cc:260] T faee3b36fdbf4f4cbba46830b3fd389f P db5f3215df92477f99cd452313a01ae0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f8926e1fc61423f9af7baa19c332c36" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38797 } } peers { permanent_uuid: "db5f3215df92477f99cd452313a01ae0" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 33837 } } peers { permanent_uuid: "1d38053e7d3f421dbb71e522b0ac13ee" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43819 } }
I20260619 02:33:41.835664 3916 raft_consensus.cc:348] T faee3b36fdbf4f4cbba46830b3fd389f P 1d38053e7d3f421dbb71e522b0ac13ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f8926e1fc61423f9af7baa19c332c36" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38797 } } peers { permanent_uuid: "db5f3215df92477f99cd452313a01ae0" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 33837 } } peers { permanent_uuid: "1d38053e7d3f421dbb71e522b0ac13ee" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43819 } }
I20260619 02:33:41.835924 3916 raft_consensus.cc:374] T faee3b36fdbf4f4cbba46830b3fd389f P 1d38053e7d3f421dbb71e522b0ac13ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:41.835969 3916 raft_consensus.cc:729] T faee3b36fdbf4f4cbba46830b3fd389f P 1d38053e7d3f421dbb71e522b0ac13ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d38053e7d3f421dbb71e522b0ac13ee, State: Initialized, Role: FOLLOWER
I20260619 02:33:41.836326 3915 ts_tablet_manager.cc:1428] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 02:33:41.836333 3914 ts_tablet_manager.cc:1428] T faee3b36fdbf4f4cbba46830b3fd389f P db5f3215df92477f99cd452313a01ae0: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 02:33:41.836403 3916 consensus_queue.cc:260] T faee3b36fdbf4f4cbba46830b3fd389f P 1d38053e7d3f421dbb71e522b0ac13ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f8926e1fc61423f9af7baa19c332c36" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38797 } } peers { permanent_uuid: "db5f3215df92477f99cd452313a01ae0" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 33837 } } peers { permanent_uuid: "1d38053e7d3f421dbb71e522b0ac13ee" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43819 } }
I20260619 02:33:41.836731 3629 heartbeater.cc:503] Master 127.17.96.254:45653 was elected leader, sending a full tablet report...
I20260619 02:33:41.836731 3897 heartbeater.cc:503] Master 127.17.96.254:45653 was elected leader, sending a full tablet report...
I20260619 02:33:41.837311 3763 heartbeater.cc:503] Master 127.17.96.254:45653 was elected leader, sending a full tablet report...
I20260619 02:33:41.837357 3916 ts_tablet_manager.cc:1428] T faee3b36fdbf4f4cbba46830b3fd389f P 1d38053e7d3f421dbb71e522b0ac13ee: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 02:33:41.894567 3920 raft_consensus.cc:482] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:33:41.894789 3920 raft_consensus.cc:504] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f8926e1fc61423f9af7baa19c332c36" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38797 } } peers { permanent_uuid: "db5f3215df92477f99cd452313a01ae0" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 33837 } } peers { permanent_uuid: "1d38053e7d3f421dbb71e522b0ac13ee" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43819 } }
I20260619 02:33:41.895970 3920 leader_election.cc:288] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db5f3215df92477f99cd452313a01ae0 (127.17.96.193:33837), 1d38053e7d3f421dbb71e522b0ac13ee (127.17.96.194:43819)
I20260619 02:33:41.899988 3911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:41.896402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:58937 (local address 127.17.96.193:33837)
0619 02:33:41.896572 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:41.896577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:41.896594 (+ 17us) server_negotiation.cc:407] Connection header received
0619 02:33:41.896881 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:41.896885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:41.896937 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:41.897043 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:41.897333 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.897888 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.898940 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.899373 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.899520 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:41.899579 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:41.899761 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 02:33:41.899844 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260619 02:33:41.899988 3899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:41.896268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:33837 (local address 127.17.96.195:58937)
0619 02:33:41.896416 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:41.896430 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:41.896580 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:41.897056 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:41.897059 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:41.897076 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:41.897211 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:41.897217 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.898008 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.898013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:41.898803 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:41.898812 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.899511 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.899515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:41.899631 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:41.899651 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 02:33:41.899743 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 02:33:41.899825 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"spinlock_wait_cycles":896}
I20260619 02:33:41.900278 3913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:41.896739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:34305 (local address 127.17.96.194:43819)
0619 02:33:41.896875 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:41.896880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:41.897060 (+ 180us) server_negotiation.cc:407] Connection header received
0619 02:33:41.897214 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:41.897217 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:41.897271 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:41.897358 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:41.897640 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.898214 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.899244 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.899670 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.899805 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:41.899855 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:41.900042 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 02:33:41.900140 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:33:41.900385 3923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:41.896611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:43819 (local address 127.17.96.195:34305)
0619 02:33:41.896963 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:41.896975 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:41.897102 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:41.897378 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:41.897381 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:41.897399 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:41.897530 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:41.897536 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.898340 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.898343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:41.899121 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:41.899129 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.899804 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.899808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:41.899917 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:41.899935 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 02:33:41.900024 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 02:33:41.900238 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":149,"threads_started":1}
I20260619 02:33:41.900815 3583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faee3b36fdbf4f4cbba46830b3fd389f" candidate_uuid: "7f8926e1fc61423f9af7baa19c332c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db5f3215df92477f99cd452313a01ae0" is_pre_election: true
I20260619 02:33:41.900998 3717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faee3b36fdbf4f4cbba46830b3fd389f" candidate_uuid: "7f8926e1fc61423f9af7baa19c332c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d38053e7d3f421dbb71e522b0ac13ee" is_pre_election: true
I20260619 02:33:41.901171 3583 raft_consensus.cc:2454] T faee3b36fdbf4f4cbba46830b3fd389f P db5f3215df92477f99cd452313a01ae0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f8926e1fc61423f9af7baa19c332c36 in term 0.
I20260619 02:33:41.901312 3717 raft_consensus.cc:2454] T faee3b36fdbf4f4cbba46830b3fd389f P 1d38053e7d3f421dbb71e522b0ac13ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f8926e1fc61423f9af7baa19c332c36 in term 0.
I20260619 02:33:41.901739 3787 leader_election.cc:302] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f8926e1fc61423f9af7baa19c332c36, db5f3215df92477f99cd452313a01ae0; no voters:
I20260619 02:33:41.902036 3920 raft_consensus.cc:2781] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:33:41.902135 3920 raft_consensus.cc:482] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:33:41.902194 3920 raft_consensus.cc:3037] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:41.903378 3920 raft_consensus.cc:504] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f8926e1fc61423f9af7baa19c332c36" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38797 } } peers { permanent_uuid: "db5f3215df92477f99cd452313a01ae0" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 33837 } } peers { permanent_uuid: "1d38053e7d3f421dbb71e522b0ac13ee" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43819 } }
I20260619 02:33:41.903781 3920 leader_election.cc:288] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36 [CANDIDATE]: Term 1 election: Requested vote from peers db5f3215df92477f99cd452313a01ae0 (127.17.96.193:33837), 1d38053e7d3f421dbb71e522b0ac13ee (127.17.96.194:43819)
I20260619 02:33:41.904181 3583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faee3b36fdbf4f4cbba46830b3fd389f" candidate_uuid: "7f8926e1fc61423f9af7baa19c332c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db5f3215df92477f99cd452313a01ae0"
I20260619 02:33:41.904218 3717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faee3b36fdbf4f4cbba46830b3fd389f" candidate_uuid: "7f8926e1fc61423f9af7baa19c332c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d38053e7d3f421dbb71e522b0ac13ee"
I20260619 02:33:41.904346 3717 raft_consensus.cc:3037] T faee3b36fdbf4f4cbba46830b3fd389f P 1d38053e7d3f421dbb71e522b0ac13ee [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:41.904340 3583 raft_consensus.cc:3037] T faee3b36fdbf4f4cbba46830b3fd389f P db5f3215df92477f99cd452313a01ae0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:41.905462 3717 raft_consensus.cc:2454] T faee3b36fdbf4f4cbba46830b3fd389f P 1d38053e7d3f421dbb71e522b0ac13ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f8926e1fc61423f9af7baa19c332c36 in term 1.
I20260619 02:33:41.905462 3583 raft_consensus.cc:2454] T faee3b36fdbf4f4cbba46830b3fd389f P db5f3215df92477f99cd452313a01ae0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f8926e1fc61423f9af7baa19c332c36 in term 1.
I20260619 02:33:41.905823 3787 leader_election.cc:302] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f8926e1fc61423f9af7baa19c332c36, db5f3215df92477f99cd452313a01ae0; no voters:
I20260619 02:33:41.906004 3920 raft_consensus.cc:2781] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:33:41.906237 3920 raft_consensus.cc:686] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36 [term 1 LEADER]: Becoming Leader. State: Replica: 7f8926e1fc61423f9af7baa19c332c36, State: Running, Role: LEADER
I20260619 02:33:41.906597 3920 consensus_queue.cc:237] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f8926e1fc61423f9af7baa19c332c36" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38797 } } peers { permanent_uuid: "db5f3215df92477f99cd452313a01ae0" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 33837 } } peers { permanent_uuid: "1d38053e7d3f421dbb71e522b0ac13ee" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43819 } }
I20260619 02:33:41.910094 3437 catalog_manager.cc:5495] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f8926e1fc61423f9af7baa19c332c36 (127.17.96.195). New cstate: current_term: 1 leader_uuid: "7f8926e1fc61423f9af7baa19c332c36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f8926e1fc61423f9af7baa19c332c36" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 38797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db5f3215df92477f99cd452313a01ae0" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 33837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d38053e7d3f421dbb71e522b0ac13ee" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43819 } health_report { overall_health: UNKNOWN } } }
I20260619 02:33:41.937911 3910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:41.933884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41712 (local address 127.17.96.195:38797)
0619 02:33:41.934034 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:41.934038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:41.934055 (+ 17us) server_negotiation.cc:407] Connection header received
0619 02:33:41.934153 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:41.934155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:41.934219 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:41.934316 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:33:41.934609 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.935284 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.936287 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:41.936505 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:41.936586 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:41.936942 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:33:41.937376 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:33:41.937497 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:41.937653 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 02:33:41.937724 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:33:41.944216 3717 raft_consensus.cc:1261] T faee3b36fdbf4f4cbba46830b3fd389f P 1d38053e7d3f421dbb71e522b0ac13ee [term 1 FOLLOWER]: Refusing update from remote peer 7f8926e1fc61423f9af7baa19c332c36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:33:41.944238 3583 raft_consensus.cc:1261] T faee3b36fdbf4f4cbba46830b3fd389f P db5f3215df92477f99cd452313a01ae0 [term 1 FOLLOWER]: Refusing update from remote peer 7f8926e1fc61423f9af7baa19c332c36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:33:41.945015 3920 consensus_queue.cc:1035] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db5f3215df92477f99cd452313a01ae0" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 33837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:33:41.945222 3924 consensus_queue.cc:1035] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d38053e7d3f421dbb71e522b0ac13ee" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 43819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:33:41.953830 3929 mvcc.cc:205] Tried to move back new op lower bound from 7298401984275673088 to 7298401984132366336. Current Snapshot: MvccSnapshot[applied={T|T < 7298401984275673088}]
I20260619 02:33:41.954416 3928 mvcc.cc:205] Tried to move back new op lower bound from 7298401984275673088 to 7298401984132366336. Current Snapshot: MvccSnapshot[applied={T|T < 7298401984275673088}]
I20260619 02:33:41.965095 3437 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:33:41.965308 3437 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:33:41.967963 3437 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 494925c62c7e4812a9de80b16dfc3d4c: Sending DeleteTablet for 3 replicas of tablet faee3b36fdbf4f4cbba46830b3fd389f
I20260619 02:33:41.968876 3831 tablet_service.cc:1514] Processing DeleteTablet for tablet faee3b36fdbf4f4cbba46830b3fd389f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:33:41 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:41696
I20260619 02:33:41.968989 3697 tablet_service.cc:1514] Processing DeleteTablet for tablet faee3b36fdbf4f4cbba46830b3fd389f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:33:41 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:40720
I20260619 02:33:41.969496 3938 tablet_replica.cc:330] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36: stopping tablet replica
I20260619 02:33:41.969534 3563 tablet_service.cc:1514] Processing DeleteTablet for tablet faee3b36fdbf4f4cbba46830b3fd389f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:33:41 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:46558
I20260619 02:33:41.970032 3938 raft_consensus.cc:2229] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:33:41.970567 3938 raft_consensus.cc:2258] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:33:41.971045 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 3500
I20260619 02:33:41.973124 3938 ts_tablet_manager.cc:1905] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:33:41.977226 3938 ts_tablet_manager.cc:1918] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:33:41.977450 3938 log.cc:1196] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/faee3b36fdbf4f4cbba46830b3fd389f
I20260619 02:33:41.977950 3938 ts_tablet_manager.cc:1939] T faee3b36fdbf4f4cbba46830b3fd389f P 7f8926e1fc61423f9af7baa19c332c36: Deleting consensus metadata
I20260619 02:33:41.978432 3939 tablet_replica.cc:330] T faee3b36fdbf4f4cbba46830b3fd389f P 1d38053e7d3f421dbb71e522b0ac13ee: stopping tablet replica
I20260619 02:33:41.978820 3939 raft_consensus.cc:2229] T faee3b36fdbf4f4cbba46830b3fd389f P 1d38053e7d3f421dbb71e522b0ac13ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:33:41.979178 3939 raft_consensus.cc:2258] T faee3b36fdbf4f4cbba46830b3fd389f P 1d38053e7d3f421dbb71e522b0ac13ee [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 02:33:41.979956 3425 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.96.193:33837 (error 108)
I20260619 02:33:41.980144 3422 catalog_manager.cc:4841] TS 7f8926e1fc61423f9af7baa19c332c36 (127.17.96.195:38797): tablet faee3b36fdbf4f4cbba46830b3fd389f (table test-table [id=7f6079c2ba2b465a95add29437e164a8]) successfully deleted
I20260619 02:33:41.980213 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 3634
I20260619 02:33:41.982436 3909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:41.982016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:33837 (local address 127.0.0.1:46560)
0619 02:33:41.982237 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:41.982324 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.193:33837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260619 02:33:41.982757 3425 catalog_manager.cc:4584] TS db5f3215df92477f99cd452313a01ae0 (127.17.96.193:33837): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet faee3b36fdbf4f4cbba46830b3fd389f: Network error: Client connection negotiation failed: client connection to 127.17.96.193:33837: connect: Connection refused (error 111)
I20260619 02:33:41.986981 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 3768
I20260619 02:33:41.989022 3909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:41.987207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:43819 (local address 127.0.0.1:40736)
0619 02:33:41.988823 (+ 1616us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:41.988905 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.194:43819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1493}
I20260619 02:33:41.993606 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 3406
2026-06-19T02:33:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2407 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-19T02:33:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:33:42Z Disabled control of system clock
I20260619 02:33:42.025282 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:44549
--webserver_interface=127.17.96.254
--webserver_port=0
--builtin_ntp_servers=127.17.96.212:38793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:44549
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:33:42.141271 3945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:42.141593 3945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:42.141633 3945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:42.145790 3945 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:33:42.145870 3945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:42.145886 3945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:42.145903 3945 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:33:42.145920 3945 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:33:42.150955 3945 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:38793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:44549
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:44549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:42.152107 3945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:42.153497 3945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:42.162572 3951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:42.162590 3950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:42.162617 3945 server_base.cc:1034] running on GCE node
W20260619 02:33:42.162575 3953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:42.163452 3945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:42.164798 3945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:42.165995 3945 hybrid_clock.cc:648] HybridClock initialized: now 1781836422165976 us; error 42 us; skew 500 ppm
I20260619 02:33:42.168033 3945 webserver.cc:460] Webserver started at http://127.17.96.254:44891/ using document root <none> and password file <none>
I20260619 02:33:42.168691 3945 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:42.168792 3945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:42.169039 3945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:42.171336 3945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "37c05fc091364239b5815afd6d0b9e0e"
format_stamp: "Formatted at 2026-06-19 02:33:42 on dist-test-slave-4qw6"
server_key: "dc46b08ca29268f2e671c39420acf21e"
server_key_iv: "14157b39f577c44a9536dd481df8b2ee"
server_key_version: "encryptionkey@0"
I20260619 02:33:42.171957 3945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "37c05fc091364239b5815afd6d0b9e0e"
format_stamp: "Formatted at 2026-06-19 02:33:42 on dist-test-slave-4qw6"
server_key: "dc46b08ca29268f2e671c39420acf21e"
server_key_iv: "14157b39f577c44a9536dd481df8b2ee"
server_key_version: "encryptionkey@0"
I20260619 02:33:42.175997 3945 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:33:42.178833 3959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:42.180321 3945 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 02:33:42.180492 3945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "37c05fc091364239b5815afd6d0b9e0e"
format_stamp: "Formatted at 2026-06-19 02:33:42 on dist-test-slave-4qw6"
server_key: "dc46b08ca29268f2e671c39420acf21e"
server_key_iv: "14157b39f577c44a9536dd481df8b2ee"
server_key_version: "encryptionkey@0"
I20260619 02:33:42.180631 3945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:42.201133 3945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:42.202059 3945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:42.202287 3945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:42.210434 3945 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:44549
I20260619 02:33:42.210451 4011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:44549 every 8 connection(s)
I20260619 02:33:42.211551 3945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:33:42.211886 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 3945
I20260619 02:33:42.212042 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:33:42.212476 17795 external_mini_cluster.cc:1428] Setting key f66c9aa688b842d8cc5be9be0a86d834
I20260619 02:33:42.215452 4012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:42.222544 4012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e: Bootstrap starting.
I20260619 02:33:42.222766 4015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:42.214184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40422 (local address 127.17.96.254:44549)
0619 02:33:42.214871 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:42.214909 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:42.214948 (+ 39us) server_negotiation.cc:407] Connection header received
0619 02:33:42.215736 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:42.215756 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:42.216185 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:42.216481 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:42.216891 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.217980 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.219230 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.219710 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.219836 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:42.220791 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:42.220796 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:42.220809 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:42.221234 (+ 425us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:42.221295 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:42.221332 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:42.221410 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:42.221588 (+ 178us) server_negotiation.cc:299] Negotiation successful
0619 02:33:42.221962 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":172,"threads_started":1}
I20260619 02:33:42.225183 4012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:42.226058 4012 log.cc:824] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:42.228096 4012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e: No bootstrap required, opened a new log
I20260619 02:33:42.231976 4012 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c05fc091364239b5815afd6d0b9e0e" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44549 } }
I20260619 02:33:42.232276 4012 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:42.232372 4012 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37c05fc091364239b5815afd6d0b9e0e, State: Initialized, Role: FOLLOWER
I20260619 02:33:42.232834 4012 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c05fc091364239b5815afd6d0b9e0e" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44549 } }
I20260619 02:33:42.232988 4012 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:33:42.233078 4012 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:33:42.233198 4012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:42.234335 4012 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c05fc091364239b5815afd6d0b9e0e" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44549 } }
I20260619 02:33:42.234714 4012 leader_election.cc:302] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37c05fc091364239b5815afd6d0b9e0e; no voters:
I20260619 02:33:42.235057 4012 leader_election.cc:288] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:33:42.235199 4017 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:33:42.235450 4017 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e [term 1 LEADER]: Becoming Leader. State: Replica: 37c05fc091364239b5815afd6d0b9e0e, State: Running, Role: LEADER
I20260619 02:33:42.235801 4017 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c05fc091364239b5815afd6d0b9e0e" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44549 } }
I20260619 02:33:42.236522 4012 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:33:42.237730 4019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37c05fc091364239b5815afd6d0b9e0e. Latest consensus state: current_term: 1 leader_uuid: "37c05fc091364239b5815afd6d0b9e0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c05fc091364239b5815afd6d0b9e0e" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44549 } } }
I20260619 02:33:42.237902 4019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:42.238209 4018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37c05fc091364239b5815afd6d0b9e0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37c05fc091364239b5815afd6d0b9e0e" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44549 } } }
I20260619 02:33:42.238301 4018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:42.238425 4026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:33:42.241685 4026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:33:42.248129 4026 catalog_manager.cc:1347] Generated new cluster ID: 596792441d8248d49e4a40e1fa129f89
I20260619 02:33:42.248232 4026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:33:42.263131 4026 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:33:42.264544 4026 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:33:42.278997 4026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e: Generated new TSK 0
I20260619 02:33:42.279979 4026 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:33:42.291306 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:44549
--builtin_ntp_servers=127.17.96.212:38793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:33:42.414587 4036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:42.414899 4036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:42.414940 4036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:42.419348 4036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:42.419430 4036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:42.419493 4036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:33:42.424557 4036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:38793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:44549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:42.425733 4036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:42.427241 4036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:42.437580 4041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:42.437641 4042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:42.437741 4036 server_base.cc:1034] running on GCE node
W20260619 02:33:42.437580 4044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:42.438416 4036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:42.439134 4036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:42.440330 4036 hybrid_clock.cc:648] HybridClock initialized: now 1781836422440314 us; error 31 us; skew 500 ppm
I20260619 02:33:42.448547 4036 webserver.cc:460] Webserver started at http://127.17.96.193:44125/ using document root <none> and password file <none>
I20260619 02:33:42.449249 4036 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:42.449357 4036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:42.449623 4036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:42.451930 4036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "00ad75061506459596c5b17b09bd7388"
format_stamp: "Formatted at 2026-06-19 02:33:42 on dist-test-slave-4qw6"
server_key: "377432b462deaebf58c401b45da74664"
server_key_iv: "1741f1993b71f7ee984045ee9d5cc376"
server_key_version: "encryptionkey@0"
I20260619 02:33:42.452574 4036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "00ad75061506459596c5b17b09bd7388"
format_stamp: "Formatted at 2026-06-19 02:33:42 on dist-test-slave-4qw6"
server_key: "377432b462deaebf58c401b45da74664"
server_key_iv: "1741f1993b71f7ee984045ee9d5cc376"
server_key_version: "encryptionkey@0"
I20260619 02:33:42.456666 4036 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 02:33:42.459517 4050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:42.460893 4036 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:33:42.461062 4036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "00ad75061506459596c5b17b09bd7388"
format_stamp: "Formatted at 2026-06-19 02:33:42 on dist-test-slave-4qw6"
server_key: "377432b462deaebf58c401b45da74664"
server_key_iv: "1741f1993b71f7ee984045ee9d5cc376"
server_key_version: "encryptionkey@0"
I20260619 02:33:42.461207 4036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:42.486191 4036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:42.486984 4036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:42.487200 4036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:42.487845 4036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:33:42.488921 4036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:33:42.489001 4036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:42.489077 4036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:33:42.489130 4036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:42.498827 4036 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:35245
I20260619 02:33:42.498881 4163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:35245 every 8 connection(s)
I20260619 02:33:42.499917 4036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:33:42.509758 4015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:42.502059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:60787 (local address 127.17.96.254:44549)
0619 02:33:42.502216 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:42.502220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:42.503415 (+ 1195us) server_negotiation.cc:407] Connection header received
0619 02:33:42.504879 (+ 1464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:42.504882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:42.504943 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:42.505025 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:42.506090 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.506618 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.507785 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.508011 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.508093 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:42.509240 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:42.509243 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:42.509245 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:42.509260 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:42.509298 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:42.509307 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:42.509366 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:42.509531 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 02:33:42.509613 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:33:42.509861 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 4036
I20260619 02:33:42.510002 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:33:42.510331 17795 external_mini_cluster.cc:1428] Setting key 1d5e189e48f4849572ee2b9e778d6c4e
I20260619 02:33:42.511057 4166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:42.502517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:44549 (local address 127.17.96.193:60787)
0619 02:33:42.503227 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:42.503273 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:42.504640 (+ 1367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:42.505210 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:42.505216 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:42.505309 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:33:42.505953 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:42.505971 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.506731 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.506733 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:42.507669 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:42.507678 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.508155 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.508158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:42.508309 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:42.508958 (+ 649us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:33:42.508960 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:33:42.509007 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:33:42.509082 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:33:42.509382 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:33:42.509389 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:33:42.509393 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:33:42.509495 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 02:33:42.509925 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":119,"threads_started":1}
I20260619 02:33:42.512434 4164 heartbeater.cc:347] Connected to a master server at 127.17.96.254:44549
I20260619 02:33:42.512616 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:44549
--builtin_ntp_servers=127.17.96.212:38793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 02:33:42.512740 4164 heartbeater.cc:464] Registering TS with master...
I20260619 02:33:42.513315 4164 heartbeater.cc:511] Master 127.17.96.254:44549 requested a full tablet report, sending...
I20260619 02:33:42.515027 3976 ts_manager.cc:194] Registered new tserver with Master: 00ad75061506459596c5b17b09bd7388 (127.17.96.193:35245)
I20260619 02:33:42.516268 3976 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.96.193:60787
W20260619 02:33:42.632265 4167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:42.632568 4167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:42.632609 4167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:42.637279 4167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:42.637353 4167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:42.637446 4167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:33:42.642921 4167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:38793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:44549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:42.644133 4167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:42.645593 4167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:42.655794 4175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:42.655797 4172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:42.655812 4173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:42.656213 4167 server_base.cc:1034] running on GCE node
I20260619 02:33:42.656684 4167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:42.657418 4167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:42.658612 4167 hybrid_clock.cc:648] HybridClock initialized: now 1781836422658565 us; error 65 us; skew 500 ppm
I20260619 02:33:42.660598 4167 webserver.cc:460] Webserver started at http://127.17.96.194:46463/ using document root <none> and password file <none>
I20260619 02:33:42.661257 4167 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:42.661377 4167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:42.661629 4167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:42.663945 4167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "8034182bb2704563b1df8c696c63ac67"
format_stamp: "Formatted at 2026-06-19 02:33:42 on dist-test-slave-4qw6"
server_key: "0bd921bc0cb13dfdeaf2010a52d672a4"
server_key_iv: "3104f5b898cdf4203046c4338285cf69"
server_key_version: "encryptionkey@0"
I20260619 02:33:42.664636 4167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "8034182bb2704563b1df8c696c63ac67"
format_stamp: "Formatted at 2026-06-19 02:33:42 on dist-test-slave-4qw6"
server_key: "0bd921bc0cb13dfdeaf2010a52d672a4"
server_key_iv: "3104f5b898cdf4203046c4338285cf69"
server_key_version: "encryptionkey@0"
I20260619 02:33:42.668915 4167 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 02:33:42.671797 4181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:42.673188 4167 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:33:42.673350 4167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "8034182bb2704563b1df8c696c63ac67"
format_stamp: "Formatted at 2026-06-19 02:33:42 on dist-test-slave-4qw6"
server_key: "0bd921bc0cb13dfdeaf2010a52d672a4"
server_key_iv: "3104f5b898cdf4203046c4338285cf69"
server_key_version: "encryptionkey@0"
I20260619 02:33:42.673496 4167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:42.689072 4167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:42.690049 4167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:42.690277 4167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:42.690970 4167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:33:42.692056 4167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:33:42.692134 4167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:42.692215 4167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:33:42.692260 4167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:42.701622 4167 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:45059
I20260619 02:33:42.701682 4294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:45059 every 8 connection(s)
I20260619 02:33:42.702710 4167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:33:42.711714 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 4167
I20260619 02:33:42.711894 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:33:42.712200 17795 external_mini_cluster.cc:1428] Setting key 21f30b96269b17d7c0d82b2078fc588e
I20260619 02:33:42.713073 4015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:42.704737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:38835 (local address 127.17.96.254:44549)
0619 02:33:42.704863 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:42.704867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:42.706130 (+ 1263us) server_negotiation.cc:407] Connection header received
0619 02:33:42.707648 (+ 1518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:42.707652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:42.707720 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:42.707806 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:42.708850 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.709409 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.710696 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.710936 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.711065 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:42.712444 (+ 1379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:42.712449 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:42.712452 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:42.712470 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:42.712496 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:42.712508 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:42.712595 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:42.712821 (+ 226us) server_negotiation.cc:299] Negotiation successful
0619 02:33:42.712917 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:33:42.714303 4297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:42.705176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:44549 (local address 127.17.96.194:38835)
0619 02:33:42.705962 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:42.706008 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:42.707392 (+ 1384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:42.708003 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:42.708010 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:42.708105 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:33:42.708674 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:42.708687 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.709575 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.709578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:42.710546 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:42.710557 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.711121 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.711125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:42.711313 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:42.712089 (+ 776us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:33:42.712091 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:33:42.712137 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:33:42.712212 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:33:42.712601 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:33:42.712608 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:33:42.712614 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:33:42.712812 (+ 198us) client_negotiation.cc:240] Negotiation successful
0619 02:33:42.713154 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":124,"threads_started":1}
I20260619 02:33:42.714738 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:44549
--builtin_ntp_servers=127.17.96.212:38793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 02:33:42.715657 4295 heartbeater.cc:347] Connected to a master server at 127.17.96.254:44549
I20260619 02:33:42.715920 4295 heartbeater.cc:464] Registering TS with master...
I20260619 02:33:42.716572 4295 heartbeater.cc:511] Master 127.17.96.254:44549 requested a full tablet report, sending...
I20260619 02:33:42.717803 3976 ts_manager.cc:194] Registered new tserver with Master: 8034182bb2704563b1df8c696c63ac67 (127.17.96.194:45059)
I20260619 02:33:42.718425 3976 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.96.194:38835
W20260619 02:33:42.832394 4298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:42.832782 4298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:42.832865 4298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:42.837395 4298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:42.837507 4298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:42.837610 4298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:33:42.842943 4298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:38793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:44549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:42.844218 4298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:42.845724 4298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:42.855985 4306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:42.856189 4298 server_base.cc:1034] running on GCE node
W20260619 02:33:42.856456 4303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:42.856489 4304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:42.856971 4298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:42.857698 4298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:42.858935 4298 hybrid_clock.cc:648] HybridClock initialized: now 1781836422858919 us; error 31 us; skew 500 ppm
I20260619 02:33:42.860977 4298 webserver.cc:460] Webserver started at http://127.17.96.195:33879/ using document root <none> and password file <none>
I20260619 02:33:42.861678 4298 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:42.861812 4298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:42.862066 4298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:42.864287 4298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "4b9d4bf5180746528ce37d04065d0f02"
format_stamp: "Formatted at 2026-06-19 02:33:42 on dist-test-slave-4qw6"
server_key: "4584d0093247a9d461ae648f649000ec"
server_key_iv: "55f3e3970e7a3028745ba53601364085"
server_key_version: "encryptionkey@0"
I20260619 02:33:42.864912 4298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "4b9d4bf5180746528ce37d04065d0f02"
format_stamp: "Formatted at 2026-06-19 02:33:42 on dist-test-slave-4qw6"
server_key: "4584d0093247a9d461ae648f649000ec"
server_key_iv: "55f3e3970e7a3028745ba53601364085"
server_key_version: "encryptionkey@0"
I20260619 02:33:42.869288 4298 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260619 02:33:42.872555 4312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:42.874024 4298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:33:42.874181 4298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "4b9d4bf5180746528ce37d04065d0f02"
format_stamp: "Formatted at 2026-06-19 02:33:42 on dist-test-slave-4qw6"
server_key: "4584d0093247a9d461ae648f649000ec"
server_key_iv: "55f3e3970e7a3028745ba53601364085"
server_key_version: "encryptionkey@0"
I20260619 02:33:42.874323 4298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:42.901157 4298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:42.902024 4298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:42.902235 4298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:42.902868 4298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:33:42.903913 4298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:33:42.903988 4298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:42.904062 4298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:33:42.904114 4298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:42.913992 4298 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:37953
I20260619 02:33:42.914054 4425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:37953 every 8 connection(s)
I20260619 02:33:42.914983 4298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:33:42.922480 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 4298
I20260619 02:33:42.922616 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:33:42.922905 17795 external_mini_cluster.cc:1428] Setting key 6faefa23186d83fe4b844ea54eba2ac6
I20260619 02:33:42.924742 4015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:42.916873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:40125 (local address 127.17.96.254:44549)
0619 02:33:42.917035 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:42.917043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:42.918181 (+ 1138us) server_negotiation.cc:407] Connection header received
0619 02:33:42.919576 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:42.919579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:42.919631 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:42.919712 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:42.920707 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.921259 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.922534 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.922891 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.923012 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:42.924172 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:42.924178 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:42.924181 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:42.924195 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:42.924220 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:42.924234 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:42.924339 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:42.924530 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 02:33:42.924599 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 02:33:42.925669 4428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:42.917291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:44549 (local address 127.17.96.195:40125)
0619 02:33:42.918025 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:42.918069 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:42.919352 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:42.919887 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:42.919894 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:42.920000 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:33:42.920523 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:42.920537 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.921378 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.921380 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:42.922383 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:42.922394 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.923075 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.923078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:42.923230 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:42.923902 (+ 672us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:33:42.923903 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:33:42.923952 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:33:42.924024 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:33:42.924342 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:33:42.924355 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:33:42.924363 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 02:33:42.924514 (+ 151us) client_negotiation.cc:240] Negotiation successful
0619 02:33:42.924825 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":114,"threads_started":1}
I20260619 02:33:42.926843 4426 heartbeater.cc:347] Connected to a master server at 127.17.96.254:44549
I20260619 02:33:42.927110 4426 heartbeater.cc:464] Registering TS with master...
I20260619 02:33:42.927673 4426 heartbeater.cc:511] Master 127.17.96.254:44549 requested a full tablet report, sending...
I20260619 02:33:42.928754 3976 ts_manager.cc:194] Registered new tserver with Master: 4b9d4bf5180746528ce37d04065d0f02 (127.17.96.195:37953)
I20260619 02:33:42.929369 3976 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.96.195:40125
I20260619 02:33:42.937469 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:33:42.943472 4015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:42.939633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54202 (local address 127.17.96.254:44549)
0619 02:33:42.939830 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:42.939834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:42.939908 (+ 74us) server_negotiation.cc:407] Connection header received
0619 02:33:42.940036 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:42.940038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:42.940109 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:42.940180 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:42.940456 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.940981 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.941836 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.942050 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.942126 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:42.942936 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:42.942939 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:42.942941 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:42.942955 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:42.942974 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:42.942982 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:42.943090 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:42.943213 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 02:33:42.943280 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260619 02:33:42.946732 3976 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54202:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:33:42.949909 3976 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:33:42.965823 4437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:42.959525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:35245 (local address 127.0.0.1:36266)
0619 02:33:42.959984 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:42.960043 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:42.960180 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:42.960832 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:42.960838 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:42.960859 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:42.961065 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:42.961076 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.962722 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.962726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:42.963950 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:42.963960 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.965141 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.965145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:42.965284 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:42.965326 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 02:33:42.965495 (+ 169us) client_negotiation.cc:240] Negotiation successful
0619 02:33:42.965630 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":116,"thread_start_us":94,"threads_started":1}
I20260619 02:33:42.965842 4440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:42.959749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36266 (local address 127.17.96.193:35245)
0619 02:33:42.960430 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:42.960436 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:42.960456 (+ 20us) server_negotiation.cc:407] Connection header received
0619 02:33:42.960524 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:42.960529 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:42.960667 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:42.961228 (+ 561us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:42.961393 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.962581 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.964096 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.964991 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.965241 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:42.965429 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:42.965491 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 02:33:42.965607 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":91,"threads_started":1}
I20260619 02:33:42.966315 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:42.959207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60026 (local address 127.17.96.194:45059)
0619 02:33:42.959573 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:42.959586 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:42.960137 (+ 551us) server_negotiation.cc:407] Connection header received
0619 02:33:42.960389 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:42.960396 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:42.960615 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:42.960750 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:42.961252 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.962550 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.964331 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.965142 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.965240 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:42.965429 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:42.966007 (+ 578us) server_negotiation.cc:299] Negotiation successful
0619 02:33:42.966127 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":118,"threads_started":1}
I20260619 02:33:42.966570 4441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:42.959792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:37953 (local address 127.0.0.1:45750)
0619 02:33:42.960451 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:42.960465 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:42.960544 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:42.961225 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:42.961228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:42.961246 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:42.961341 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:42.961346 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.962528 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.962534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:42.964139 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:42.964151 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.965112 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.965116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:42.966233 (+ 1117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:42.966259 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 02:33:42.966333 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 02:33:42.966420 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":71,"thread_start_us":98,"threads_started":1}
I20260619 02:33:42.967010 4442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:42.960240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45750 (local address 127.17.96.195:37953)
0619 02:33:42.960768 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:42.960779 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:42.960801 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:33:42.960874 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:42.960879 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:42.961066 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:42.961466 (+ 400us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:42.961575 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.962394 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.964352 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.964972 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.965111 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:42.965343 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:42.966526 (+ 1183us) server_negotiation.cc:299] Negotiation successful
0619 02:33:42.966816 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":163,"threads_started":1}
I20260619 02:33:42.967095 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:42.959048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:45059 (local address 127.0.0.1:60026)
0619 02:33:42.959986 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:42.960042 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:42.960222 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:42.960982 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:42.960985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:42.961000 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:42.961130 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:42.961137 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.962675 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.962678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:42.964139 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:42.964151 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:42.965589 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:42.965592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:42.965675 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:42.965708 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 02:33:42.966537 (+ 829us) client_negotiation.cc:240] Negotiation successful
0619 02:33:42.966618 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"mutex_wait_us":151,"thread_start_us":95,"threads_started":1}
I20260619 02:33:42.969693 4360 tablet_service.cc:1467] Processing CreateTablet for tablet c1f775c706f84c7fb5ad5d7c6b5d6820 (DEFAULT_TABLE table=test-table [id=4f18f75eca3740a7b6b4987b62ce6a33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:33:42.969703 4098 tablet_service.cc:1467] Processing CreateTablet for tablet c1f775c706f84c7fb5ad5d7c6b5d6820 (DEFAULT_TABLE table=test-table [id=4f18f75eca3740a7b6b4987b62ce6a33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:33:42.970041 4229 tablet_service.cc:1467] Processing CreateTablet for tablet c1f775c706f84c7fb5ad5d7c6b5d6820 (DEFAULT_TABLE table=test-table [id=4f18f75eca3740a7b6b4987b62ce6a33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:33:42.970937 4098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1f775c706f84c7fb5ad5d7c6b5d6820. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:42.970937 4229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1f775c706f84c7fb5ad5d7c6b5d6820. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:42.970950 4360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1f775c706f84c7fb5ad5d7c6b5d6820. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:42.978245 4443 tablet_bootstrap.cc:492] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02: Bootstrap starting.
I20260619 02:33:42.979660 4445 tablet_bootstrap.cc:492] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 00ad75061506459596c5b17b09bd7388: Bootstrap starting.
I20260619 02:33:42.981355 4443 tablet_bootstrap.cc:654] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:42.981626 4445 tablet_bootstrap.cc:654] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 00ad75061506459596c5b17b09bd7388: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:42.982225 4444 tablet_bootstrap.cc:492] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 8034182bb2704563b1df8c696c63ac67: Bootstrap starting.
I20260619 02:33:42.982303 4443 log.cc:824] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:42.982597 4445 log.cc:824] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 00ad75061506459596c5b17b09bd7388: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:42.984706 4445 tablet_bootstrap.cc:492] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 00ad75061506459596c5b17b09bd7388: No bootstrap required, opened a new log
I20260619 02:33:42.984723 4443 tablet_bootstrap.cc:492] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02: No bootstrap required, opened a new log
I20260619 02:33:42.984709 4444 tablet_bootstrap.cc:654] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 8034182bb2704563b1df8c696c63ac67: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:42.984937 4443 ts_tablet_manager.cc:1397] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260619 02:33:42.984977 4445 ts_tablet_manager.cc:1397] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 00ad75061506459596c5b17b09bd7388: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260619 02:33:42.985693 4444 log.cc:824] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 8034182bb2704563b1df8c696c63ac67: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:42.988741 4443 raft_consensus.cc:348] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8034182bb2704563b1df8c696c63ac67" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 45059 } } peers { permanent_uuid: "4b9d4bf5180746528ce37d04065d0f02" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 37953 } } peers { permanent_uuid: "00ad75061506459596c5b17b09bd7388" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35245 } }
I20260619 02:33:42.988740 4445 raft_consensus.cc:348] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 00ad75061506459596c5b17b09bd7388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8034182bb2704563b1df8c696c63ac67" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 45059 } } peers { permanent_uuid: "4b9d4bf5180746528ce37d04065d0f02" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 37953 } } peers { permanent_uuid: "00ad75061506459596c5b17b09bd7388" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35245 } }
I20260619 02:33:42.988992 4443 raft_consensus.cc:374] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:42.989024 4445 raft_consensus.cc:374] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 00ad75061506459596c5b17b09bd7388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:42.989027 4443 raft_consensus.cc:729] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b9d4bf5180746528ce37d04065d0f02, State: Initialized, Role: FOLLOWER
I20260619 02:33:42.989101 4445 raft_consensus.cc:729] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 00ad75061506459596c5b17b09bd7388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00ad75061506459596c5b17b09bd7388, State: Initialized, Role: FOLLOWER
I20260619 02:33:42.989565 4443 consensus_queue.cc:260] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8034182bb2704563b1df8c696c63ac67" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 45059 } } peers { permanent_uuid: "4b9d4bf5180746528ce37d04065d0f02" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 37953 } } peers { permanent_uuid: "00ad75061506459596c5b17b09bd7388" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35245 } }
I20260619 02:33:42.989571 4445 consensus_queue.cc:260] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 00ad75061506459596c5b17b09bd7388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8034182bb2704563b1df8c696c63ac67" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 45059 } } peers { permanent_uuid: "4b9d4bf5180746528ce37d04065d0f02" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 37953 } } peers { permanent_uuid: "00ad75061506459596c5b17b09bd7388" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35245 } }
I20260619 02:33:42.990581 4164 heartbeater.cc:503] Master 127.17.96.254:44549 was elected leader, sending a full tablet report...
I20260619 02:33:42.990702 4445 ts_tablet_manager.cc:1428] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 00ad75061506459596c5b17b09bd7388: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260619 02:33:42.990706 4443 ts_tablet_manager.cc:1428] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260619 02:33:42.991186 4444 tablet_bootstrap.cc:492] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 8034182bb2704563b1df8c696c63ac67: No bootstrap required, opened a new log
I20260619 02:33:42.991348 4444 ts_tablet_manager.cc:1397] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 8034182bb2704563b1df8c696c63ac67: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260619 02:33:42.991683 4426 heartbeater.cc:503] Master 127.17.96.254:44549 was elected leader, sending a full tablet report...
I20260619 02:33:42.995707 4444 raft_consensus.cc:348] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 8034182bb2704563b1df8c696c63ac67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8034182bb2704563b1df8c696c63ac67" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 45059 } } peers { permanent_uuid: "4b9d4bf5180746528ce37d04065d0f02" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 37953 } } peers { permanent_uuid: "00ad75061506459596c5b17b09bd7388" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35245 } }
I20260619 02:33:42.996001 4444 raft_consensus.cc:374] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 8034182bb2704563b1df8c696c63ac67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:42.996086 4444 raft_consensus.cc:729] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 8034182bb2704563b1df8c696c63ac67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8034182bb2704563b1df8c696c63ac67, State: Initialized, Role: FOLLOWER
I20260619 02:33:42.996539 4444 consensus_queue.cc:260] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 8034182bb2704563b1df8c696c63ac67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8034182bb2704563b1df8c696c63ac67" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 45059 } } peers { permanent_uuid: "4b9d4bf5180746528ce37d04065d0f02" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 37953 } } peers { permanent_uuid: "00ad75061506459596c5b17b09bd7388" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35245 } }
I20260619 02:33:42.997248 4295 heartbeater.cc:503] Master 127.17.96.254:44549 was elected leader, sending a full tablet report...
I20260619 02:33:42.997542 4444 ts_tablet_manager.cc:1428] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 8034182bb2704563b1df8c696c63ac67: Time spent starting tablet: real 0.006s user 0.004s sys 0.004s
W20260619 02:33:43.001212 4165 tablet.cc:2369] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 00ad75061506459596c5b17b09bd7388: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:33:43.127180 4450 raft_consensus.cc:482] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:33:43.127460 4450 raft_consensus.cc:504] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8034182bb2704563b1df8c696c63ac67" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 45059 } } peers { permanent_uuid: "4b9d4bf5180746528ce37d04065d0f02" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 37953 } } peers { permanent_uuid: "00ad75061506459596c5b17b09bd7388" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35245 } }
I20260619 02:33:43.128769 4450 leader_election.cc:288] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8034182bb2704563b1df8c696c63ac67 (127.17.96.194:45059), 00ad75061506459596c5b17b09bd7388 (127.17.96.193:35245)
I20260619 02:33:43.132776 4428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:43.129052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:35245 (local address 127.17.96.195:33261)
0619 02:33:43.129284 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:43.129302 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:43.129410 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:43.129710 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:43.129713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:43.129729 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:43.129892 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:43.129899 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:43.130707 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:43.130711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:43.131582 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:43.131592 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:43.132286 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:43.132290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:43.132391 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:43.132421 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 02:33:43.132491 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 02:33:43.132590 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":1}
I20260619 02:33:43.132787 4440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:43.129166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:33261 (local address 127.17.96.193:35245)
0619 02:33:43.129292 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:43.129298 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:43.129385 (+ 87us) server_negotiation.cc:407] Connection header received
0619 02:33:43.129523 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:43.129529 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:43.129600 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:43.129681 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:43.130006 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:43.130589 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:43.131720 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:43.132142 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:43.132243 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:43.132332 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:43.132540 (+ 208us) server_negotiation.cc:299] Negotiation successful
0619 02:33:43.132625 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 02:33:43.132813 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:43.129123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:55609 (local address 127.17.96.194:45059)
0619 02:33:43.129285 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:43.129291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:43.129634 (+ 343us) server_negotiation.cc:407] Connection header received
0619 02:33:43.129720 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:43.129723 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:43.129804 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:43.129888 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:43.130199 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:43.130721 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:43.131795 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:43.132211 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:43.132330 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:43.132383 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:43.132600 (+ 217us) server_negotiation.cc:299] Negotiation successful
0619 02:33:43.132692 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 02:33:43.132787 4452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:43.129059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:45059 (local address 127.17.96.195:55609)
0619 02:33:43.129519 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:43.129532 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:43.129640 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:43.129914 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:43.129916 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:43.129930 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:43.130081 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:43.130088 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:43.130843 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:43.130845 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:43.131631 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:43.131669 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:43.132358 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:43.132363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:43.132448 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:43.132464 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 02:33:43.132564 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 02:33:43.132625 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":49,"thread_start_us":111,"threads_started":1}
I20260619 02:33:43.133546 4118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1f775c706f84c7fb5ad5d7c6b5d6820" candidate_uuid: "4b9d4bf5180746528ce37d04065d0f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00ad75061506459596c5b17b09bd7388" is_pre_election: true
I20260619 02:33:43.133721 4249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1f775c706f84c7fb5ad5d7c6b5d6820" candidate_uuid: "4b9d4bf5180746528ce37d04065d0f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8034182bb2704563b1df8c696c63ac67" is_pre_election: true
I20260619 02:33:43.133917 4118 raft_consensus.cc:2454] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 00ad75061506459596c5b17b09bd7388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b9d4bf5180746528ce37d04065d0f02 in term 0.
I20260619 02:33:43.134105 4249 raft_consensus.cc:2454] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 8034182bb2704563b1df8c696c63ac67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b9d4bf5180746528ce37d04065d0f02 in term 0.
I20260619 02:33:43.134589 4314 leader_election.cc:302] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 00ad75061506459596c5b17b09bd7388, 4b9d4bf5180746528ce37d04065d0f02, 8034182bb2704563b1df8c696c63ac67; no voters:
I20260619 02:33:43.134903 4450 raft_consensus.cc:2781] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:33:43.134989 4450 raft_consensus.cc:482] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:33:43.135031 4450 raft_consensus.cc:3037] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:43.136129 4450 raft_consensus.cc:504] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8034182bb2704563b1df8c696c63ac67" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 45059 } } peers { permanent_uuid: "4b9d4bf5180746528ce37d04065d0f02" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 37953 } } peers { permanent_uuid: "00ad75061506459596c5b17b09bd7388" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35245 } }
I20260619 02:33:43.136495 4450 leader_election.cc:288] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02 [CANDIDATE]: Term 1 election: Requested vote from peers 8034182bb2704563b1df8c696c63ac67 (127.17.96.194:45059), 00ad75061506459596c5b17b09bd7388 (127.17.96.193:35245)
I20260619 02:33:43.136869 4249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1f775c706f84c7fb5ad5d7c6b5d6820" candidate_uuid: "4b9d4bf5180746528ce37d04065d0f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8034182bb2704563b1df8c696c63ac67"
I20260619 02:33:43.136909 4118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1f775c706f84c7fb5ad5d7c6b5d6820" candidate_uuid: "4b9d4bf5180746528ce37d04065d0f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00ad75061506459596c5b17b09bd7388"
I20260619 02:33:43.137037 4118 raft_consensus.cc:3037] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 00ad75061506459596c5b17b09bd7388 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:43.137017 4249 raft_consensus.cc:3037] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 8034182bb2704563b1df8c696c63ac67 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:43.138330 4118 raft_consensus.cc:2454] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 00ad75061506459596c5b17b09bd7388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b9d4bf5180746528ce37d04065d0f02 in term 1.
I20260619 02:33:43.138443 4249 raft_consensus.cc:2454] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 8034182bb2704563b1df8c696c63ac67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b9d4bf5180746528ce37d04065d0f02 in term 1.
I20260619 02:33:43.138669 4314 leader_election.cc:302] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00ad75061506459596c5b17b09bd7388, 4b9d4bf5180746528ce37d04065d0f02; no voters:
I20260619 02:33:43.138919 4450 raft_consensus.cc:2781] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:33:43.139147 4450 raft_consensus.cc:686] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02 [term 1 LEADER]: Becoming Leader. State: Replica: 4b9d4bf5180746528ce37d04065d0f02, State: Running, Role: LEADER
I20260619 02:33:43.139513 4450 consensus_queue.cc:237] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8034182bb2704563b1df8c696c63ac67" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 45059 } } peers { permanent_uuid: "4b9d4bf5180746528ce37d04065d0f02" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 37953 } } peers { permanent_uuid: "00ad75061506459596c5b17b09bd7388" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35245 } }
I20260619 02:33:43.143062 3975 catalog_manager.cc:5495] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b9d4bf5180746528ce37d04065d0f02 (127.17.96.195). New cstate: current_term: 1 leader_uuid: "4b9d4bf5180746528ce37d04065d0f02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8034182bb2704563b1df8c696c63ac67" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 45059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b9d4bf5180746528ce37d04065d0f02" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 37953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00ad75061506459596c5b17b09bd7388" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35245 } health_report { overall_health: UNKNOWN } } }
I20260619 02:33:43.162799 4442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:43.158812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45752 (local address 127.17.96.195:37953)
0619 02:33:43.158957 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:43.158960 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:43.158978 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:33:43.159095 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:43.159098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:43.159161 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:43.159267 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:33:43.159483 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:43.160086 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:43.160962 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:43.161181 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:43.161286 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:43.161541 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:33:43.162146 (+ 605us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:33:43.162341 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:43.162533 (+ 192us) server_negotiation.cc:299] Negotiation successful
0619 02:33:43.162576 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260619 02:33:43.165974 4427 tablet.cc:2369] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:33:43.169579 4249 raft_consensus.cc:1261] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 8034182bb2704563b1df8c696c63ac67 [term 1 FOLLOWER]: Refusing update from remote peer 4b9d4bf5180746528ce37d04065d0f02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:33:43.169579 4118 raft_consensus.cc:1261] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 00ad75061506459596c5b17b09bd7388 [term 1 FOLLOWER]: Refusing update from remote peer 4b9d4bf5180746528ce37d04065d0f02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:33:43.170455 4453 consensus_queue.cc:1035] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00ad75061506459596c5b17b09bd7388" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 35245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:33:43.170673 4450 consensus_queue.cc:1035] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8034182bb2704563b1df8c696c63ac67" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:33:43.189937 3975 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:33:43.190162 3975 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:33:43.194303 3975 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 37c05fc091364239b5815afd6d0b9e0e: Sending DeleteTablet for 3 replicas of tablet c1f775c706f84c7fb5ad5d7c6b5d6820
I20260619 02:33:43.195338 4098 tablet_service.cc:1514] Processing DeleteTablet for tablet c1f775c706f84c7fb5ad5d7c6b5d6820 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:33:43 UTC) from {username='slave'} at 127.0.0.1:36266
I20260619 02:33:43.195412 4360 tablet_service.cc:1514] Processing DeleteTablet for tablet c1f775c706f84c7fb5ad5d7c6b5d6820 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:33:43 UTC) from {username='slave'} at 127.0.0.1:45750
I20260619 02:33:43.195977 4467 tablet_replica.cc:330] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02: stopping tablet replica
I20260619 02:33:43.196214 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 4036
I20260619 02:33:43.196178 4229 tablet_service.cc:1514] Processing DeleteTablet for tablet c1f775c706f84c7fb5ad5d7c6b5d6820 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:33:43 UTC) from {username='slave'} at 127.0.0.1:60026
I20260619 02:33:43.196379 4467 raft_consensus.cc:2229] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:33:43.196816 4467 raft_consensus.cc:2258] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:33:43.197150 4468 tablet_replica.cc:330] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 8034182bb2704563b1df8c696c63ac67: stopping tablet replica
I20260619 02:33:43.197481 4468 raft_consensus.cc:2229] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 8034182bb2704563b1df8c696c63ac67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:33:43.197849 4468 raft_consensus.cc:2258] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 8034182bb2704563b1df8c696c63ac67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:33:43.199064 4467 ts_tablet_manager.cc:1905] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:33:43.199079 4468 ts_tablet_manager.cc:1905] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 8034182bb2704563b1df8c696c63ac67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:33:43.202003 4468 ts_tablet_manager.cc:1918] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 8034182bb2704563b1df8c696c63ac67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:33:43.202003 4467 ts_tablet_manager.cc:1918] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:33:43.202121 4468 log.cc:1196] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 8034182bb2704563b1df8c696c63ac67: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/c1f775c706f84c7fb5ad5d7c6b5d6820
I20260619 02:33:43.202121 4467 log.cc:1196] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/c1f775c706f84c7fb5ad5d7c6b5d6820
I20260619 02:33:43.202451 4467 ts_tablet_manager.cc:1939] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 4b9d4bf5180746528ce37d04065d0f02: Deleting consensus metadata
I20260619 02:33:43.202458 4468 ts_tablet_manager.cc:1939] T c1f775c706f84c7fb5ad5d7c6b5d6820 P 8034182bb2704563b1df8c696c63ac67: Deleting consensus metadata
I20260619 02:33:43.203817 3960 catalog_manager.cc:4841] TS 8034182bb2704563b1df8c696c63ac67 (127.17.96.194:45059): tablet c1f775c706f84c7fb5ad5d7c6b5d6820 (table test-table [id=4f18f75eca3740a7b6b4987b62ce6a33]) successfully deleted
I20260619 02:33:43.204056 3960 catalog_manager.cc:4841] TS 4b9d4bf5180746528ce37d04065d0f02 (127.17.96.195:37953): tablet c1f775c706f84c7fb5ad5d7c6b5d6820 (table test-table [id=4f18f75eca3740a7b6b4987b62ce6a33]) successfully deleted
W20260619 02:33:43.207073 3961 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.96.193:35245 (error 108)
I20260619 02:33:43.207255 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 4167
I20260619 02:33:43.214975 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 4298
I20260619 02:33:43.215050 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:43.214720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:35245 (local address 127.0.0.1:36276)
0619 02:33:43.214881 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:43.214956 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.193:35245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260619 02:33:43.216553 3961 catalog_manager.cc:4584] TS 00ad75061506459596c5b17b09bd7388 (127.17.96.193:35245): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c1f775c706f84c7fb5ad5d7c6b5d6820: Network error: Client connection negotiation failed: client connection to 127.17.96.193:35245: connect: Connection refused (error 111)
I20260619 02:33:43.221732 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 3945
2026-06-19T02:33:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1227 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:33:43 dist-test-slave-4qw6 krb5kdc[4472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:33:43 dist-test-slave-4qw6 krb5kdc[4472](info): set up 2 sockets
Jun 19 02:33:43 dist-test-slave-4qw6 krb5kdc[4472](info): commencing operation
krb5kdc: starting...
W20260619 02:33:44.278278 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:33:44 dist-test-slave-4qw6 krb5kdc[4472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:33:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:33:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:33:44.409005 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:41257
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:35303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:41257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:33:44.528390 4485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:44.528827 4485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:44.528924 4485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:44.533319 4485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:33:44.533483 4485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:44.533527 4485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:44.533571 4485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:33:44.533627 4485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:33:44.539454 4485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:35303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:41257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:41257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:44.540745 4485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:44.541921 4485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:44.550983 4491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:44.550983 4490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:44.551277 4485 server_base.cc:1034] running on GCE node
W20260619 02:33:44.550987 4493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:44.552099 4485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:44.553359 4485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:44.554566 4485 hybrid_clock.cc:648] HybridClock initialized: now 1781836424554544 us; error 37 us; skew 500 ppm
Jun 19 02:33:44 dist-test-slave-4qw6 krb5kdc[4472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836424, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:44.557516 4485 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:33:44.558524 4485 webserver.cc:460] Webserver started at http://127.17.96.254:39133/ using document root <none> and password file <none>
I20260619 02:33:44.559077 4485 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:44.559136 4485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:44.559309 4485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:44.561472 4485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "f6c55d4655754b429921891cd78a161e"
format_stamp: "Formatted at 2026-06-19 02:33:44 on dist-test-slave-4qw6"
server_key: "71bd2240724fa10b1d4980caee1b8b60"
server_key_iv: "fb29db1bace1a9515bb1e737a4f05c39"
server_key_version: "encryptionkey@0"
I20260619 02:33:44.562103 4485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "f6c55d4655754b429921891cd78a161e"
format_stamp: "Formatted at 2026-06-19 02:33:44 on dist-test-slave-4qw6"
server_key: "71bd2240724fa10b1d4980caee1b8b60"
server_key_iv: "fb29db1bace1a9515bb1e737a4f05c39"
server_key_version: "encryptionkey@0"
I20260619 02:33:44.566010 4485 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260619 02:33:44.568638 4500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:44.570111 4485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:33:44.570235 4485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "f6c55d4655754b429921891cd78a161e"
format_stamp: "Formatted at 2026-06-19 02:33:44 on dist-test-slave-4qw6"
server_key: "71bd2240724fa10b1d4980caee1b8b60"
server_key_iv: "fb29db1bace1a9515bb1e737a4f05c39"
server_key_version: "encryptionkey@0"
I20260619 02:33:44.570349 4485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:44.591130 4485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:44.595203 4485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:44.595480 4485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:44.605038 4485 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:41257
I20260619 02:33:44.605239 4552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:41257 every 8 connection(s)
I20260619 02:33:44.606357 4485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:33:44.606631 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 4485
I20260619 02:33:44.606814 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:33:44.607215 17795 external_mini_cluster.cc:1428] Setting key 5b97086a58658b213763aae0c431a14a
I20260619 02:33:44.609560 4553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:44.617138 4553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e: Bootstrap starting.
Jun 19 02:33:44 dist-test-slave-4qw6 krb5kdc[4472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:33:44.620131 4553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:44.621140 4553 log.cc:824] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:44.623847 4553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e: No bootstrap required, opened a new log
I20260619 02:33:44.624668 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:44.608738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47104 (local address 127.17.96.254:41257)
0619 02:33:44.609229 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:44.609253 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:44.609493 (+ 240us) server_negotiation.cc:407] Connection header received
0619 02:33:44.610611 (+ 1118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:44.610634 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:44.611103 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:44.611499 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:44.612298 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:44.613968 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:44.614980 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:44.615709 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:44.615897 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:44.618945 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:44.618953 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:44.618970 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:33:44.619010 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:44.621180 (+ 2170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:44.621826 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:44.621833 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:44.621840 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:44.621985 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:44.622331 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:44.622334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:44.622336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:44.622855 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:44.623070 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:44.623327 (+ 257us) server_negotiation.cc:299] Negotiation successful
0619 02:33:44.623650 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":170,"threads_started":1}
I20260619 02:33:44.628243 4553 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6c55d4655754b429921891cd78a161e" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 41257 } }
I20260619 02:33:44.628532 4553 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:44.628572 4553 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6c55d4655754b429921891cd78a161e, State: Initialized, Role: FOLLOWER
I20260619 02:33:44.628966 4553 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6c55d4655754b429921891cd78a161e" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 41257 } }
I20260619 02:33:44.629084 4553 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:33:44.629128 4553 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:33:44.629202 4553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:44.630326 4553 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6c55d4655754b429921891cd78a161e" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 41257 } }
I20260619 02:33:44.630679 4553 leader_election.cc:302] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6c55d4655754b429921891cd78a161e; no voters:
I20260619 02:33:44.631014 4553 leader_election.cc:288] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:33:44.631140 4558 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:33:44.631433 4558 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e [term 1 LEADER]: Becoming Leader. State: Replica: f6c55d4655754b429921891cd78a161e, State: Running, Role: LEADER
I20260619 02:33:44.631807 4558 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6c55d4655754b429921891cd78a161e" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 41257 } }
I20260619 02:33:44.632325 4553 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:33:44.633560 4560 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6c55d4655754b429921891cd78a161e. Latest consensus state: current_term: 1 leader_uuid: "f6c55d4655754b429921891cd78a161e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6c55d4655754b429921891cd78a161e" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 41257 } } }
I20260619 02:33:44.633721 4560 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:44.634158 4559 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6c55d4655754b429921891cd78a161e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6c55d4655754b429921891cd78a161e" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 41257 } } }
I20260619 02:33:44.634254 4567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:33:44.634347 4559 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:44.637611 4567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:33:44.643873 4567 catalog_manager.cc:1347] Generated new cluster ID: 623a3e82caf64712ae8e82f3e0ab18ae
I20260619 02:33:44.644014 4567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:33:44.661127 4567 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:33:44.662528 4567 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:33:44.669286 4567 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e: Generated new TSK 0
I20260619 02:33:44.670192 4567 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.96.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
Entry for principal kudu/127.17.96.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab.
I20260619 02:33:44.705534 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:41257
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:35303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:33:44.837888 4579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:44.838240 4579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:44.838320 4579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:44.844328 4579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:44.844413 4579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:44.844508 4579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:33:44.852581 4579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:35303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.193.keytab
--principal=kudu/127.17.96.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:41257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:44.854277 4579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:44.855806 4579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:44.868736 4585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:44.868716 4584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:44.869025 4587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:44.869024 4579 server_base.cc:1034] running on GCE node
I20260619 02:33:44.869653 4579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:44.870709 4579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:44.871912 4579 hybrid_clock.cc:648] HybridClock initialized: now 1781836424871839 us; error 108 us; skew 500 ppm
Jun 19 02:33:44 dist-test-slave-4qw6 krb5kdc[4472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836424, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:44.876101 4579 init.cc:377] Logged in from keytab as kudu/127.17.96.193@KRBTEST.COM (short username kudu)
I20260619 02:33:44.877493 4579 webserver.cc:460] Webserver started at http://127.17.96.193:44873/ using document root <none> and password file <none>
I20260619 02:33:44.878244 4579 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:44.878352 4579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:44.878592 4579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:44.881680 4579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "3a375e103884434c95bfb26afb66a27f"
format_stamp: "Formatted at 2026-06-19 02:33:44 on dist-test-slave-4qw6"
server_key: "544cd896056debbd7225d1c173368ee1"
server_key_iv: "1fb2ed19cacab4bd1af0668664cc8858"
server_key_version: "encryptionkey@0"
I20260619 02:33:44.882444 4579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "3a375e103884434c95bfb26afb66a27f"
format_stamp: "Formatted at 2026-06-19 02:33:44 on dist-test-slave-4qw6"
server_key: "544cd896056debbd7225d1c173368ee1"
server_key_iv: "1fb2ed19cacab4bd1af0668664cc8858"
server_key_version: "encryptionkey@0"
I20260619 02:33:44.886855 4579 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 02:33:44.889814 4594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:44.891324 4579 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:33:44.891491 4579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "3a375e103884434c95bfb26afb66a27f"
format_stamp: "Formatted at 2026-06-19 02:33:44 on dist-test-slave-4qw6"
server_key: "544cd896056debbd7225d1c173368ee1"
server_key_iv: "1fb2ed19cacab4bd1af0668664cc8858"
server_key_version: "encryptionkey@0"
I20260619 02:33:44.891636 4579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:44.939252 4579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:44.943037 4579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:44.943296 4579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:44.944023 4579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:33:44.945017 4579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:33:44.945098 4579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:44.945180 4579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:33:44.945245 4579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:44.956172 4579 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:34189
I20260619 02:33:44.956192 4707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:34189 every 8 connection(s)
I20260619 02:33:44.957378 4579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:33:44.964296 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 4579
I20260619 02:33:44.964483 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:33:44.964809 17795 external_mini_cluster.cc:1428] Setting key 7e66f2bc2f47c197580ffbeb591ca4cb
Jun 19 02:33:44 dist-test-slave-4qw6 krb5kdc[4472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836424, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.193@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:33:44.974846 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:44.959661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:53137 (local address 127.17.96.254:41257)
0619 02:33:44.959862 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:44.959866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:44.960832 (+ 966us) server_negotiation.cc:407] Connection header received
0619 02:33:44.962502 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:44.962506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:44.962580 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:44.962709 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:44.964118 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:44.964810 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:44.966396 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:44.966741 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:44.966859 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:44.970203 (+ 3344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:44.970207 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:44.970209 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:33:44.970239 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:44.971967 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:44.972782 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:44.972784 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:44.972786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:44.972870 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:44.973344 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:44.973346 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:44.973347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:44.973624 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:44.973707 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:44.974561 (+ 854us) server_negotiation.cc:299] Negotiation successful
0619 02:33:44.974710 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
WARNING: no policy specified for kudu/127.17.96.194@KRBTEST.COM; defaulting to no policy
I20260619 02:33:44.976202 4710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:44.960089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:41257 (local address 127.17.96.193:53137)
0619 02:33:44.960616 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:44.960694 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:44.962235 (+ 1541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:44.962917 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:44.962925 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:44.963352 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:33:44.963920 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:44.963936 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:44.964991 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:44.964995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:44.966197 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:44.966209 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:44.966919 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:44.966923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:44.967099 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:44.967843 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:33:44.967862 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:33:44.969973 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:33:44.972121 (+ 2148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:33:44.972131 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:33:44.972155 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:33:44.972649 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:33:44.972970 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:33:44.972975 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:33:44.972978 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:33:44.973213 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:33:44.973779 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:33:44.973788 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:33:44.974211 (+ 423us) client_negotiation.cc:769] Sending connection context
0619 02:33:44.974480 (+ 269us) client_negotiation.cc:240] Negotiation successful
0619 02:33:44.974904 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":146,"threads_started":1}
I20260619 02:33:44.977684 4708 heartbeater.cc:347] Connected to a master server at 127.17.96.254:41257
I20260619 02:33:44.978000 4708 heartbeater.cc:464] Registering TS with master...
I20260619 02:33:44.978615 4708 heartbeater.cc:511] Master 127.17.96.254:41257 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.194@KRBTEST.COM" created.
I20260619 02:33:44.980453 4514 ts_manager.cc:194] Registered new tserver with Master: 3a375e103884434c95bfb26afb66a27f (127.17.96.193:34189)
I20260619 02:33:44.981710 4514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.193@KRBTEST.COM'} at 127.17.96.193:53137
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
Entry for principal kudu/127.17.96.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab.
I20260619 02:33:44.997378 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:41257
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:35303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:33:45.119207 4713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:45.119549 4713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:45.119599 4713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:45.123893 4713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:45.124019 4713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:45.124128 4713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:33:45.130214 4713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:35303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.194.keytab
--principal=kudu/127.17.96.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:41257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:45.131551 4713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:45.132838 4713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:45.143999 4718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:45.144024 4719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:45.144001 4721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:45.144400 4713 server_base.cc:1034] running on GCE node
I20260619 02:33:45.144904 4713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:45.145654 4713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:45.146869 4713 hybrid_clock.cc:648] HybridClock initialized: now 1781836425146856 us; error 34 us; skew 500 ppm
Jun 19 02:33:45 dist-test-slave-4qw6 krb5kdc[4472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836425, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:45.150040 4713 init.cc:377] Logged in from keytab as kudu/127.17.96.194@KRBTEST.COM (short username kudu)
I20260619 02:33:45.151074 4713 webserver.cc:460] Webserver started at http://127.17.96.194:35161/ using document root <none> and password file <none>
I20260619 02:33:45.151710 4713 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:45.151814 4713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:45.152065 4713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:45.154459 4713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "9749dd5fe8a246459371b131a134b0bd"
format_stamp: "Formatted at 2026-06-19 02:33:45 on dist-test-slave-4qw6"
server_key: "9a3347654bf71805b7ef6756c7629a1e"
server_key_iv: "16114567fbd76834062a0f2677ac82e7"
server_key_version: "encryptionkey@0"
I20260619 02:33:45.155128 4713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "9749dd5fe8a246459371b131a134b0bd"
format_stamp: "Formatted at 2026-06-19 02:33:45 on dist-test-slave-4qw6"
server_key: "9a3347654bf71805b7ef6756c7629a1e"
server_key_iv: "16114567fbd76834062a0f2677ac82e7"
server_key_version: "encryptionkey@0"
I20260619 02:33:45.159809 4713 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:33:45.162685 4728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:45.164278 4713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:33:45.164491 4713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "9749dd5fe8a246459371b131a134b0bd"
format_stamp: "Formatted at 2026-06-19 02:33:45 on dist-test-slave-4qw6"
server_key: "9a3347654bf71805b7ef6756c7629a1e"
server_key_iv: "16114567fbd76834062a0f2677ac82e7"
server_key_version: "encryptionkey@0"
I20260619 02:33:45.164671 4713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:45.178521 4713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:45.182453 4713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:45.182708 4713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:45.183415 4713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:33:45.184406 4713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:33:45.184487 4713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:45.184558 4713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:33:45.184605 4713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:45.194648 4713 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:46125
I20260619 02:33:45.194691 4841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:46125 every 8 connection(s)
I20260619 02:33:45.195747 4713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:33:45.205341 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 4713
I20260619 02:33:45.205518 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:33:45.205945 17795 external_mini_cluster.cc:1428] Setting key b0196d4f61dd322f9dc54d7ced48b034
Jun 19 02:33:45 dist-test-slave-4qw6 krb5kdc[4472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836425, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.194@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:33:45.211505 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:45.197876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:36521 (local address 127.17.96.254:41257)
0619 02:33:45.198043 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:45.198047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:45.199014 (+ 967us) server_negotiation.cc:407] Connection header received
0619 02:33:45.200400 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:45.200403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:45.200458 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:45.200538 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:45.201861 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.202418 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.203676 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.203927 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.204043 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:45.207207 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:45.207211 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:45.207219 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:33:45.207270 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:45.208985 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:45.209589 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:45.209594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:45.209595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:45.209695 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:45.210116 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:45.210119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:45.210121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:45.210330 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:45.210454 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:45.211036 (+ 582us) server_negotiation.cc:299] Negotiation successful
0619 02:33:45.211182 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 02:33:45.212670 4844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:45.198306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:41257 (local address 127.17.96.194:36521)
0619 02:33:45.198818 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:45.198868 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:45.200164 (+ 1296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:45.200722 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:45.200729 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:45.201137 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:33:45.201654 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:45.201669 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.202569 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.202572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:45.203516 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:45.203527 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.204098 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.204101 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:45.204262 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:45.204981 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:33:45.205000 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:33:45.207014 (+ 2014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:33:45.209130 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:33:45.209137 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:33:45.209157 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:33:45.209469 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:33:45.209826 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:33:45.209829 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:33:45.209831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:33:45.210008 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:33:45.210456 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:33:45.210466 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:33:45.210767 (+ 301us) client_negotiation.cc:769] Sending connection context
0619 02:33:45.210991 (+ 224us) client_negotiation.cc:240] Negotiation successful
0619 02:33:45.211512 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":152,"threads_started":1}
I20260619 02:33:45.214265 4842 heartbeater.cc:347] Connected to a master server at 127.17.96.254:41257
I20260619 02:33:45.214608 4842 heartbeater.cc:464] Registering TS with master...
I20260619 02:33:45.215365 4842 heartbeater.cc:511] Master 127.17.96.254:41257 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.96.195@KRBTEST.COM; defaulting to no policy
I20260619 02:33:45.216969 4514 ts_manager.cc:194] Registered new tserver with Master: 9749dd5fe8a246459371b131a134b0bd (127.17.96.194:46125)
I20260619 02:33:45.217590 4514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.194@KRBTEST.COM'} at 127.17.96.194:36521
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
Entry for principal kudu/127.17.96.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab.
I20260619 02:33:45.236804 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:41257
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:35303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:33:45.357033 4847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:45.357375 4847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:45.357424 4847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:45.362344 4847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:45.362447 4847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:45.362519 4847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:33:45.369589 4847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:35303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.195.keytab
--principal=kudu/127.17.96.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:41257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:45.371012 4847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:45.372212 4847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:45.384102 4855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:45.384092 4853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:45.384104 4852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:45.384639 4847 server_base.cc:1034] running on GCE node
I20260619 02:33:45.385161 4847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:45.386041 4847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:45.387251 4847 hybrid_clock.cc:648] HybridClock initialized: now 1781836425387228 us; error 60 us; skew 500 ppm
Jun 19 02:33:45 dist-test-slave-4qw6 krb5kdc[4472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836425, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:45.391222 4847 init.cc:377] Logged in from keytab as kudu/127.17.96.195@KRBTEST.COM (short username kudu)
I20260619 02:33:45.392588 4847 webserver.cc:460] Webserver started at http://127.17.96.195:41853/ using document root <none> and password file <none>
I20260619 02:33:45.393304 4847 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:45.393376 4847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:45.393599 4847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:45.396565 4847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "4ff356e3fb344ade8147e480c6c9f15f"
format_stamp: "Formatted at 2026-06-19 02:33:45 on dist-test-slave-4qw6"
server_key: "3d65b98a4d13effcfe288eb6bed62585"
server_key_iv: "0f47fd3660431997ea79c9502301b98e"
server_key_version: "encryptionkey@0"
I20260619 02:33:45.397300 4847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "4ff356e3fb344ade8147e480c6c9f15f"
format_stamp: "Formatted at 2026-06-19 02:33:45 on dist-test-slave-4qw6"
server_key: "3d65b98a4d13effcfe288eb6bed62585"
server_key_iv: "0f47fd3660431997ea79c9502301b98e"
server_key_version: "encryptionkey@0"
I20260619 02:33:45.402954 4847 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 02:33:45.406229 4862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:45.407770 4847 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 02:33:45.407972 4847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "4ff356e3fb344ade8147e480c6c9f15f"
format_stamp: "Formatted at 2026-06-19 02:33:45 on dist-test-slave-4qw6"
server_key: "3d65b98a4d13effcfe288eb6bed62585"
server_key_iv: "0f47fd3660431997ea79c9502301b98e"
server_key_version: "encryptionkey@0"
I20260619 02:33:45.408138 4847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:45.436530 4847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:45.440524 4847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:45.440783 4847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:45.441488 4847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:33:45.442584 4847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:33:45.442667 4847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:45.442750 4847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:33:45.442800 4847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:45.453655 4847 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:36355
I20260619 02:33:45.453681 4975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:36355 every 8 connection(s)
I20260619 02:33:45.454778 4847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:33:45.455432 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 4847
I20260619 02:33:45.455629 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:33:45.456169 17795 external_mini_cluster.cc:1428] Setting key 174f93a06739c5d6d402a49c94fc0faf
Jun 19 02:33:45 dist-test-slave-4qw6 krb5kdc[4472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836425, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.195@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:33:45.473878 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:45.457180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:53823 (local address 127.17.96.254:41257)
0619 02:33:45.457370 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:45.457375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:45.458537 (+ 1162us) server_negotiation.cc:407] Connection header received
0619 02:33:45.460365 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:45.460368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:45.460426 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:45.460504 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:45.462144 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.462829 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.464314 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.464603 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.464742 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:45.468781 (+ 4039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:45.468786 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:45.468788 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:33:45.468836 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:45.470828 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:45.471682 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:45.471685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:45.471686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:45.471802 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:45.472396 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:45.472419 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:45.472420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:45.472650 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:45.472779 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:45.473503 (+ 724us) server_negotiation.cc:299] Negotiation successful
0619 02:33:45.473665 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 02:33:45.474897 4978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:45.457613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:41257 (local address 127.17.96.195:53823)
0619 02:33:45.458323 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:45.458394 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:45.460123 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:45.460710 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:45.460719 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:45.461176 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:33:45.461840 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:45.461860 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.463005 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.463009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:45.464093 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:45.464107 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.464916 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.464935 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:45.465211 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:45.466251 (+ 1040us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:33:45.466273 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:33:45.468483 (+ 2210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:33:45.471061 (+ 2578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:33:45.471073 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:33:45.471099 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:33:45.471525 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:33:45.471983 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:33:45.471985 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:33:45.471987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:33:45.472273 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:33:45.472779 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:33:45.472789 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:33:45.473174 (+ 385us) client_negotiation.cc:769] Sending connection context
0619 02:33:45.473431 (+ 257us) client_negotiation.cc:240] Negotiation successful
0619 02:33:45.473885 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":203,"threads_started":1}
I20260619 02:33:45.476424 4976 heartbeater.cc:347] Connected to a master server at 127.17.96.254:41257
I20260619 02:33:45.476756 4976 heartbeater.cc:464] Registering TS with master...
I20260619 02:33:45.477334 4976 heartbeater.cc:511] Master 127.17.96.254:41257 requested a full tablet report, sending...
I20260619 02:33:45.478681 4514 ts_manager.cc:194] Registered new tserver with Master: 4ff356e3fb344ade8147e480c6c9f15f (127.17.96.195:36355)
I20260619 02:33:45.479287 4514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.96.195@KRBTEST.COM'} at 127.17.96.195:53823
I20260619 02:33:45.484776 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:33:45.496850 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:45.487424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47106 (local address 127.17.96.254:41257)
0619 02:33:45.487707 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:45.487715 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:45.487984 (+ 269us) server_negotiation.cc:407] Connection header received
0619 02:33:45.488241 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:45.488245 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:45.488313 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:45.488532 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:45.489471 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.490160 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.491150 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.491404 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.491537 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:45.493220 (+ 1683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:45.493223 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:45.493226 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:33:45.493258 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:45.494802 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:45.495309 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:45.495314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:45.495316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:45.495463 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:45.495787 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:45.495791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:45.495793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:45.496070 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:45.496240 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:45.496515 (+ 275us) server_negotiation.cc:299] Negotiation successful
0619 02:33:45.496660 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260619 02:33:45.500447 4514 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47106:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:33:45.503834 4514 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:33:45.519997 4992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:45.514317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35302 (local address 127.17.96.193:34189)
0619 02:33:45.514737 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:45.514745 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:45.514770 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:33:45.515006 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:45.515011 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:45.515192 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:45.515337 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:45.515553 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.516671 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.518227 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.519145 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.519264 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:45.519447 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:45.519702 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 02:33:45.519824 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":119,"threads_started":1}
I20260619 02:33:45.519987 4987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:45.513029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:46125 (local address 127.0.0.1:39198)
0619 02:33:45.513570 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:45.513622 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:45.513796 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:45.515595 (+ 1799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:45.515598 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:45.515614 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:45.515698 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:45.515703 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.517112 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.517115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:45.518500 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:45.518511 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.519593 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.519599 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:45.519701 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:45.519721 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 02:33:45.519785 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 02:33:45.519838 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":3,"thread_start_us":89,"threads_started":1}
I20260619 02:33:45.520545 4988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:45.513182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39198 (local address 127.17.96.194:46125)
0619 02:33:45.515056 (+ 1874us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:45.515068 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:45.515089 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:33:45.515156 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:45.515161 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:45.515318 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:45.515443 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:45.515835 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.516935 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.518634 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.519271 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.519383 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:45.519527 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:45.520284 (+ 757us) server_negotiation.cc:299] Negotiation successful
0619 02:33:45.520401 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1766,"thread_start_us":133,"threads_started":1}
I20260619 02:33:45.520676 4990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:45.513742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54096 (local address 127.17.96.195:36355)
0619 02:33:45.514145 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:45.514157 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:45.514522 (+ 365us) server_negotiation.cc:407] Connection header received
0619 02:33:45.514612 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:45.514617 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:45.514874 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:45.515074 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:45.515409 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.516323 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.518686 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.519540 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.519692 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:45.519823 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:45.520411 (+ 588us) server_negotiation.cc:299] Negotiation successful
0619 02:33:45.520538 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":178,"threads_started":1}
I20260619 02:33:45.519987 4989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:45.514114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:34189 (local address 127.0.0.1:35302)
0619 02:33:45.514446 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:45.514699 (+ 253us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:45.514797 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:45.515331 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:45.515335 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:45.515356 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:45.515459 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:45.515467 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.516795 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.516799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:45.518090 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:45.518102 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.519290 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.519295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:45.519441 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:45.519484 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 02:33:45.519614 (+ 130us) client_negotiation.cc:240] Negotiation successful
0619 02:33:45.519795 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":77,"thread_start_us":109,"threads_started":1}
I20260619 02:33:45.520326 4991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:45.513220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:36355 (local address 127.0.0.1:54096)
0619 02:33:45.514371 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:45.514385 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:45.514490 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:45.515061 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:45.515067 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:45.515091 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:45.515246 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:45.515253 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.516473 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.516479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:45.518090 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:45.518102 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.519903 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.519907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:45.520002 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:45.520039 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 02:33:45.520115 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 02:33:45.520199 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1081,"mutex_wait_us":75,"thread_start_us":123,"threads_started":1}
I20260619 02:33:45.524014 4910 tablet_service.cc:1467] Processing CreateTablet for tablet 6b75d532a1a04096afe24b90c606ea5c (DEFAULT_TABLE table=test-table [id=121ac4e008c342a0bc119b2aeff8c3a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:33:45.524091 4776 tablet_service.cc:1467] Processing CreateTablet for tablet 6b75d532a1a04096afe24b90c606ea5c (DEFAULT_TABLE table=test-table [id=121ac4e008c342a0bc119b2aeff8c3a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:33:45.524091 4642 tablet_service.cc:1467] Processing CreateTablet for tablet 6b75d532a1a04096afe24b90c606ea5c (DEFAULT_TABLE table=test-table [id=121ac4e008c342a0bc119b2aeff8c3a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:33:45.525287 4910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b75d532a1a04096afe24b90c606ea5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:45.525286 4642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b75d532a1a04096afe24b90c606ea5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:45.525286 4776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b75d532a1a04096afe24b90c606ea5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:45.533258 4993 tablet_bootstrap.cc:492] T 6b75d532a1a04096afe24b90c606ea5c P 4ff356e3fb344ade8147e480c6c9f15f: Bootstrap starting.
I20260619 02:33:45.534646 4995 tablet_bootstrap.cc:492] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd: Bootstrap starting.
I20260619 02:33:45.535140 4994 tablet_bootstrap.cc:492] T 6b75d532a1a04096afe24b90c606ea5c P 3a375e103884434c95bfb26afb66a27f: Bootstrap starting.
I20260619 02:33:45.536691 4993 tablet_bootstrap.cc:654] T 6b75d532a1a04096afe24b90c606ea5c P 4ff356e3fb344ade8147e480c6c9f15f: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:45.536731 4995 tablet_bootstrap.cc:654] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:45.537218 4994 tablet_bootstrap.cc:654] T 6b75d532a1a04096afe24b90c606ea5c P 3a375e103884434c95bfb26afb66a27f: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:45.538082 4995 log.cc:824] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:45.538112 4994 log.cc:824] T 6b75d532a1a04096afe24b90c606ea5c P 3a375e103884434c95bfb26afb66a27f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:45.538112 4993 log.cc:824] T 6b75d532a1a04096afe24b90c606ea5c P 4ff356e3fb344ade8147e480c6c9f15f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:45.540161 4994 tablet_bootstrap.cc:492] T 6b75d532a1a04096afe24b90c606ea5c P 3a375e103884434c95bfb26afb66a27f: No bootstrap required, opened a new log
I20260619 02:33:45.540256 4995 tablet_bootstrap.cc:492] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd: No bootstrap required, opened a new log
I20260619 02:33:45.540459 4994 ts_tablet_manager.cc:1397] T 6b75d532a1a04096afe24b90c606ea5c P 3a375e103884434c95bfb26afb66a27f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 02:33:45.540472 4995 ts_tablet_manager.cc:1397] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 02:33:45.540534 4993 tablet_bootstrap.cc:492] T 6b75d532a1a04096afe24b90c606ea5c P 4ff356e3fb344ade8147e480c6c9f15f: No bootstrap required, opened a new log
I20260619 02:33:45.540726 4993 ts_tablet_manager.cc:1397] T 6b75d532a1a04096afe24b90c606ea5c P 4ff356e3fb344ade8147e480c6c9f15f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260619 02:33:45.544348 4994 raft_consensus.cc:348] T 6b75d532a1a04096afe24b90c606ea5c P 3a375e103884434c95bfb26afb66a27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9749dd5fe8a246459371b131a134b0bd" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 46125 } } peers { permanent_uuid: "4ff356e3fb344ade8147e480c6c9f15f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36355 } } peers { permanent_uuid: "3a375e103884434c95bfb26afb66a27f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 34189 } }
I20260619 02:33:45.544718 4994 raft_consensus.cc:374] T 6b75d532a1a04096afe24b90c606ea5c P 3a375e103884434c95bfb26afb66a27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:45.544782 4994 raft_consensus.cc:729] T 6b75d532a1a04096afe24b90c606ea5c P 3a375e103884434c95bfb26afb66a27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a375e103884434c95bfb26afb66a27f, State: Initialized, Role: FOLLOWER
I20260619 02:33:45.544577 4995 raft_consensus.cc:348] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9749dd5fe8a246459371b131a134b0bd" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 46125 } } peers { permanent_uuid: "4ff356e3fb344ade8147e480c6c9f15f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36355 } } peers { permanent_uuid: "3a375e103884434c95bfb26afb66a27f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 34189 } }
I20260619 02:33:45.544922 4995 raft_consensus.cc:374] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:45.544993 4995 raft_consensus.cc:729] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9749dd5fe8a246459371b131a134b0bd, State: Initialized, Role: FOLLOWER
I20260619 02:33:45.545332 4994 consensus_queue.cc:260] T 6b75d532a1a04096afe24b90c606ea5c P 3a375e103884434c95bfb26afb66a27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9749dd5fe8a246459371b131a134b0bd" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 46125 } } peers { permanent_uuid: "4ff356e3fb344ade8147e480c6c9f15f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36355 } } peers { permanent_uuid: "3a375e103884434c95bfb26afb66a27f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 34189 } }
I20260619 02:33:45.545346 4993 raft_consensus.cc:348] T 6b75d532a1a04096afe24b90c606ea5c P 4ff356e3fb344ade8147e480c6c9f15f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9749dd5fe8a246459371b131a134b0bd" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 46125 } } peers { permanent_uuid: "4ff356e3fb344ade8147e480c6c9f15f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36355 } } peers { permanent_uuid: "3a375e103884434c95bfb26afb66a27f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 34189 } }
I20260619 02:33:45.545426 4995 consensus_queue.cc:260] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9749dd5fe8a246459371b131a134b0bd" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 46125 } } peers { permanent_uuid: "4ff356e3fb344ade8147e480c6c9f15f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36355 } } peers { permanent_uuid: "3a375e103884434c95bfb26afb66a27f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 34189 } }
I20260619 02:33:45.545583 4993 raft_consensus.cc:374] T 6b75d532a1a04096afe24b90c606ea5c P 4ff356e3fb344ade8147e480c6c9f15f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:45.545646 4993 raft_consensus.cc:729] T 6b75d532a1a04096afe24b90c606ea5c P 4ff356e3fb344ade8147e480c6c9f15f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ff356e3fb344ade8147e480c6c9f15f, State: Initialized, Role: FOLLOWER
I20260619 02:33:45.546197 4842 heartbeater.cc:503] Master 127.17.96.254:41257 was elected leader, sending a full tablet report...
I20260619 02:33:45.546139 4993 consensus_queue.cc:260] T 6b75d532a1a04096afe24b90c606ea5c P 4ff356e3fb344ade8147e480c6c9f15f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9749dd5fe8a246459371b131a134b0bd" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 46125 } } peers { permanent_uuid: "4ff356e3fb344ade8147e480c6c9f15f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36355 } } peers { permanent_uuid: "3a375e103884434c95bfb26afb66a27f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 34189 } }
I20260619 02:33:45.546589 4994 ts_tablet_manager.cc:1428] T 6b75d532a1a04096afe24b90c606ea5c P 3a375e103884434c95bfb26afb66a27f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260619 02:33:45.546847 4976 heartbeater.cc:503] Master 127.17.96.254:41257 was elected leader, sending a full tablet report...
I20260619 02:33:45.547056 4993 ts_tablet_manager.cc:1428] T 6b75d532a1a04096afe24b90c606ea5c P 4ff356e3fb344ade8147e480c6c9f15f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260619 02:33:45.547755 4995 ts_tablet_manager.cc:1428] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260619 02:33:45.548076 4708 heartbeater.cc:503] Master 127.17.96.254:41257 was elected leader, sending a full tablet report...
W20260619 02:33:45.697157 4843 tablet.cc:2369] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:33:45.706295 4977 tablet.cc:2369] T 6b75d532a1a04096afe24b90c606ea5c P 4ff356e3fb344ade8147e480c6c9f15f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:33:45.709088 4709 tablet.cc:2369] T 6b75d532a1a04096afe24b90c606ea5c P 3a375e103884434c95bfb26afb66a27f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:33:45.805662 5000 raft_consensus.cc:482] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:33:45.806034 5000 raft_consensus.cc:504] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9749dd5fe8a246459371b131a134b0bd" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 46125 } } peers { permanent_uuid: "4ff356e3fb344ade8147e480c6c9f15f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36355 } } peers { permanent_uuid: "3a375e103884434c95bfb26afb66a27f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 34189 } }
I20260619 02:33:45.807516 5000 leader_election.cc:288] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ff356e3fb344ade8147e480c6c9f15f (127.17.96.195:36355), 3a375e103884434c95bfb26afb66a27f (127.17.96.193:34189)
I20260619 02:33:45.811830 4990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:45.808123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:39773 (local address 127.17.96.195:36355)
0619 02:33:45.808272 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:45.808277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:45.808295 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:33:45.808367 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:45.808374 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:45.808457 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:45.808544 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:45.808872 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.809485 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.810701 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.811158 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.811264 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:45.811326 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:45.811593 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 02:33:45.811697 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 02:33:45.811849 5002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:45.807727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:36355 (local address 127.17.96.194:39773)
0619 02:33:45.808090 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:45.808107 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:45.808208 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:45.808575 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:45.808578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:45.808599 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:45.808748 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:45.808756 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.809621 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.809624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:45.810563 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:45.810575 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.811320 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.811324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:45.811441 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:45.811471 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 02:33:45.811559 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 02:33:45.811643 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":84,"threads_started":1}
I20260619 02:33:45.811910 5003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:45.807776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:34189 (local address 127.17.96.194:48179)
0619 02:33:45.808280 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:45.808298 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:45.808452 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:45.808762 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:45.808764 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:45.808780 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:45.808941 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:45.808950 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.809832 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.809841 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:45.810660 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:45.810673 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.811451 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.811453 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:45.811582 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:45.811607 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 02:33:45.811685 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 02:33:45.811754 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
I20260619 02:33:45.811930 4992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:45.807933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:48179 (local address 127.17.96.193:34189)
0619 02:33:45.808083 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:45.808090 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:45.808409 (+ 319us) server_negotiation.cc:407] Connection header received
0619 02:33:45.808578 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:45.808581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:45.808630 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:45.808730 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:45.809070 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.809626 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.810796 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.811311 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.811468 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:45.811534 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:45.811718 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 02:33:45.811802 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:33:45.812647 4930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b75d532a1a04096afe24b90c606ea5c" candidate_uuid: "9749dd5fe8a246459371b131a134b0bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ff356e3fb344ade8147e480c6c9f15f" is_pre_election: true
I20260619 02:33:45.812747 4662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b75d532a1a04096afe24b90c606ea5c" candidate_uuid: "9749dd5fe8a246459371b131a134b0bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a375e103884434c95bfb26afb66a27f" is_pre_election: true
I20260619 02:33:45.813024 4662 raft_consensus.cc:2454] T 6b75d532a1a04096afe24b90c606ea5c P 3a375e103884434c95bfb26afb66a27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9749dd5fe8a246459371b131a134b0bd in term 0.
I20260619 02:33:45.813024 4930 raft_consensus.cc:2454] T 6b75d532a1a04096afe24b90c606ea5c P 4ff356e3fb344ade8147e480c6c9f15f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9749dd5fe8a246459371b131a134b0bd in term 0.
I20260619 02:33:45.813668 4731 leader_election.cc:302] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3a375e103884434c95bfb26afb66a27f, 4ff356e3fb344ade8147e480c6c9f15f, 9749dd5fe8a246459371b131a134b0bd; no voters:
I20260619 02:33:45.814041 5000 raft_consensus.cc:2781] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:33:45.814137 5000 raft_consensus.cc:482] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:33:45.814191 5000 raft_consensus.cc:3037] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:45.815325 5000 raft_consensus.cc:504] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9749dd5fe8a246459371b131a134b0bd" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 46125 } } peers { permanent_uuid: "4ff356e3fb344ade8147e480c6c9f15f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36355 } } peers { permanent_uuid: "3a375e103884434c95bfb26afb66a27f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 34189 } }
I20260619 02:33:45.815723 5000 leader_election.cc:288] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd [CANDIDATE]: Term 1 election: Requested vote from peers 4ff356e3fb344ade8147e480c6c9f15f (127.17.96.195:36355), 3a375e103884434c95bfb26afb66a27f (127.17.96.193:34189)
I20260619 02:33:45.816200 4930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b75d532a1a04096afe24b90c606ea5c" candidate_uuid: "9749dd5fe8a246459371b131a134b0bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ff356e3fb344ade8147e480c6c9f15f"
I20260619 02:33:45.816200 4662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b75d532a1a04096afe24b90c606ea5c" candidate_uuid: "9749dd5fe8a246459371b131a134b0bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a375e103884434c95bfb26afb66a27f"
I20260619 02:33:45.816359 4662 raft_consensus.cc:3037] T 6b75d532a1a04096afe24b90c606ea5c P 3a375e103884434c95bfb26afb66a27f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:45.816359 4930 raft_consensus.cc:3037] T 6b75d532a1a04096afe24b90c606ea5c P 4ff356e3fb344ade8147e480c6c9f15f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:45.817838 4662 raft_consensus.cc:2454] T 6b75d532a1a04096afe24b90c606ea5c P 3a375e103884434c95bfb26afb66a27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9749dd5fe8a246459371b131a134b0bd in term 1.
I20260619 02:33:45.817837 4930 raft_consensus.cc:2454] T 6b75d532a1a04096afe24b90c606ea5c P 4ff356e3fb344ade8147e480c6c9f15f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9749dd5fe8a246459371b131a134b0bd in term 1.
I20260619 02:33:45.818281 4729 leader_election.cc:302] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3a375e103884434c95bfb26afb66a27f, 4ff356e3fb344ade8147e480c6c9f15f, 9749dd5fe8a246459371b131a134b0bd; no voters:
I20260619 02:33:45.818480 5000 raft_consensus.cc:2781] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:33:45.818722 5000 raft_consensus.cc:686] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd [term 1 LEADER]: Becoming Leader. State: Replica: 9749dd5fe8a246459371b131a134b0bd, State: Running, Role: LEADER
I20260619 02:33:45.819144 5000 consensus_queue.cc:237] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9749dd5fe8a246459371b131a134b0bd" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 46125 } } peers { permanent_uuid: "4ff356e3fb344ade8147e480c6c9f15f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36355 } } peers { permanent_uuid: "3a375e103884434c95bfb26afb66a27f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 34189 } }
I20260619 02:33:45.822635 4517 catalog_manager.cc:5495] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd reported cstate change: term changed from 0 to 1, leader changed from <none> to 9749dd5fe8a246459371b131a134b0bd (127.17.96.194). New cstate: current_term: 1 leader_uuid: "9749dd5fe8a246459371b131a134b0bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9749dd5fe8a246459371b131a134b0bd" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 46125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ff356e3fb344ade8147e480c6c9f15f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a375e103884434c95bfb26afb66a27f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 34189 } health_report { overall_health: UNKNOWN } } }
I20260619 02:33:45.888103 4988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:45.884200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39202 (local address 127.17.96.194:46125)
0619 02:33:45.884354 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:45.884357 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:45.884407 (+ 50us) server_negotiation.cc:407] Connection header received
0619 02:33:45.884490 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:45.884493 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:45.884556 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:45.884645 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:33:45.884883 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.885508 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.886518 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:45.886804 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:45.886898 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:45.887196 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:33:45.887670 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:33:45.887769 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:45.887844 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 02:33:45.887910 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 02:33:45.894975 4930 raft_consensus.cc:1261] T 6b75d532a1a04096afe24b90c606ea5c P 4ff356e3fb344ade8147e480c6c9f15f [term 1 FOLLOWER]: Refusing update from remote peer 9749dd5fe8a246459371b131a134b0bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:33:45.894975 4662 raft_consensus.cc:1261] T 6b75d532a1a04096afe24b90c606ea5c P 3a375e103884434c95bfb26afb66a27f [term 1 FOLLOWER]: Refusing update from remote peer 9749dd5fe8a246459371b131a134b0bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:33:45.895857 5004 consensus_queue.cc:1035] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ff356e3fb344ade8147e480c6c9f15f" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 36355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:33:45.896099 5000 consensus_queue.cc:1035] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a375e103884434c95bfb26afb66a27f" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 34189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:33:45.907203 5009 mvcc.cc:205] Tried to move back new op lower bound from 7298402000456994816 to 7298402000158072832. Current Snapshot: MvccSnapshot[applied={T|T < 7298402000456994816}]
I20260619 02:33:45.907999 5008 mvcc.cc:205] Tried to move back new op lower bound from 7298402000456994816 to 7298402000158072832. Current Snapshot: MvccSnapshot[applied={T|T < 7298402000456994816 or (T in {7298402000456994816})}]
I20260619 02:33:45.914619 4515 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:33:45.914845 4515 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:33:45.917559 4515 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6c55d4655754b429921891cd78a161e: Sending DeleteTablet for 3 replicas of tablet 6b75d532a1a04096afe24b90c606ea5c
I20260619 02:33:45.918629 4910 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b75d532a1a04096afe24b90c606ea5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:33:45 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:54096
I20260619 02:33:45.918607 4776 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b75d532a1a04096afe24b90c606ea5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:33:45 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:39198
I20260619 02:33:45.918751 4642 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b75d532a1a04096afe24b90c606ea5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:33:45 UTC) from {username='kudu', principal='kudu/127.17.96.254@KRBTEST.COM'} at 127.0.0.1:35302
I20260619 02:33:45.919327 5018 tablet_replica.cc:330] T 6b75d532a1a04096afe24b90c606ea5c P 3a375e103884434c95bfb26afb66a27f: stopping tablet replica
I20260619 02:33:45.919474 5017 tablet_replica.cc:330] T 6b75d532a1a04096afe24b90c606ea5c P 4ff356e3fb344ade8147e480c6c9f15f: stopping tablet replica
I20260619 02:33:45.919780 5017 raft_consensus.cc:2229] T 6b75d532a1a04096afe24b90c606ea5c P 4ff356e3fb344ade8147e480c6c9f15f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:33:45.919781 5018 raft_consensus.cc:2229] T 6b75d532a1a04096afe24b90c606ea5c P 3a375e103884434c95bfb26afb66a27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:33:45.920128 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 4579
I20260619 02:33:45.920962 5019 tablet_replica.cc:330] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd: stopping tablet replica
I20260619 02:33:45.921388 5019 raft_consensus.cc:2229] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:33:45.921960 5019 raft_consensus.cc:2258] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:33:45.922957 5017 raft_consensus.cc:2258] T 6b75d532a1a04096afe24b90c606ea5c P 4ff356e3fb344ade8147e480c6c9f15f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:33:45.923879 5019 ts_tablet_manager.cc:1905] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:33:45.924539 5017 ts_tablet_manager.cc:1905] T 6b75d532a1a04096afe24b90c606ea5c P 4ff356e3fb344ade8147e480c6c9f15f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:33:45.927279 5019 ts_tablet_manager.cc:1918] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:33:45.927461 5017 ts_tablet_manager.cc:1918] T 6b75d532a1a04096afe24b90c606ea5c P 4ff356e3fb344ade8147e480c6c9f15f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:33:45.927564 5017 log.cc:1196] T 6b75d532a1a04096afe24b90c606ea5c P 4ff356e3fb344ade8147e480c6c9f15f: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/6b75d532a1a04096afe24b90c606ea5c
I20260619 02:33:45.927500 5019 log.cc:1196] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/6b75d532a1a04096afe24b90c606ea5c
I20260619 02:33:45.927919 5017 ts_tablet_manager.cc:1939] T 6b75d532a1a04096afe24b90c606ea5c P 4ff356e3fb344ade8147e480c6c9f15f: Deleting consensus metadata
I20260619 02:33:45.928095 5019 ts_tablet_manager.cc:1939] T 6b75d532a1a04096afe24b90c606ea5c P 9749dd5fe8a246459371b131a134b0bd: Deleting consensus metadata
W20260619 02:33:45.929025 4503 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.96.193:34189 (error 108)
I20260619 02:33:45.929423 4504 catalog_manager.cc:4841] TS 9749dd5fe8a246459371b131a134b0bd (127.17.96.194:46125): tablet 6b75d532a1a04096afe24b90c606ea5c (table test-table [id=121ac4e008c342a0bc119b2aeff8c3a3]) successfully deleted
I20260619 02:33:45.929675 4501 catalog_manager.cc:4841] TS 4ff356e3fb344ade8147e480c6c9f15f (127.17.96.195:36355): tablet 6b75d532a1a04096afe24b90c606ea5c (table test-table [id=121ac4e008c342a0bc119b2aeff8c3a3]) successfully deleted
I20260619 02:33:45.930300 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 4713
I20260619 02:33:45.931077 4991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:45.930201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:34189 (local address 127.0.0.1:35316)
0619 02:33:45.930872 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:45.930963 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.193:34189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":561}
W20260619 02:33:45.932267 4503 catalog_manager.cc:4584] TS 3a375e103884434c95bfb26afb66a27f (127.17.96.193:34189): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6b75d532a1a04096afe24b90c606ea5c: Network error: Client connection negotiation failed: client connection to 127.17.96.193:34189: connect: Connection refused (error 111)
I20260619 02:33:45.936426 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 4847
I20260619 02:33:45.942901 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 4485
2026-06-19T02:33:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2722 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-19T02:33:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:33:45Z Disabled control of system clock
I20260619 02:33:45.975708 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:37363
--webserver_interface=127.17.96.254
--webserver_port=0
--builtin_ntp_servers=127.17.96.212:45553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:37363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:33:46.102941 5024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:46.103287 5024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:46.103331 5024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:46.109131 5024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:33:46.109243 5024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:46.109269 5024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:46.109300 5024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:33:46.109336 5024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:33:46.114897 5024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:45553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:37363
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:37363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:46.116156 5024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:46.117640 5024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:46.126958 5032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:46.126933 5029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:46.126947 5030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:46.127607 5024 server_base.cc:1034] running on GCE node
I20260619 02:33:46.128259 5024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:46.130091 5024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:46.131313 5024 hybrid_clock.cc:648] HybridClock initialized: now 1781836426131308 us; error 40 us; skew 500 ppm
I20260619 02:33:46.133680 5024 webserver.cc:460] Webserver started at http://127.17.96.254:40749/ using document root <none> and password file <none>
I20260619 02:33:46.134440 5024 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:46.134563 5024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:46.134853 5024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:46.137370 5024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "b6b6202ba4df477fae8021cb6d306149"
format_stamp: "Formatted at 2026-06-19 02:33:46 on dist-test-slave-4qw6"
server_key: "735836fa4359e87bb3502b5173bbd506"
server_key_iv: "a1147b24b5b737e96748f79a02c7119c"
server_key_version: "encryptionkey@0"
I20260619 02:33:46.138087 5024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "b6b6202ba4df477fae8021cb6d306149"
format_stamp: "Formatted at 2026-06-19 02:33:46 on dist-test-slave-4qw6"
server_key: "735836fa4359e87bb3502b5173bbd506"
server_key_iv: "a1147b24b5b737e96748f79a02c7119c"
server_key_version: "encryptionkey@0"
I20260619 02:33:46.142555 5024 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 02:33:46.145349 5038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:46.146880 5024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 02:33:46.147070 5024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "b6b6202ba4df477fae8021cb6d306149"
format_stamp: "Formatted at 2026-06-19 02:33:46 on dist-test-slave-4qw6"
server_key: "735836fa4359e87bb3502b5173bbd506"
server_key_iv: "a1147b24b5b737e96748f79a02c7119c"
server_key_version: "encryptionkey@0"
I20260619 02:33:46.147212 5024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:46.173398 5024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:46.174309 5024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:46.174575 5024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:46.182816 5024 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:37363
I20260619 02:33:46.182842 5090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:37363 every 8 connection(s)
I20260619 02:33:46.183935 5024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:33:46.187505 5091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:46.193531 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 5024
I20260619 02:33:46.193668 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:33:46.194033 17795 external_mini_cluster.cc:1428] Setting key 59721cd06973c251997a017b5991ff2c
I20260619 02:33:46.194176 5091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149: Bootstrap starting.
I20260619 02:33:46.197201 5091 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:46.198191 5091 log.cc:824] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:46.200423 5091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149: No bootstrap required, opened a new log
I20260619 02:33:46.205928 5094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:46.195779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41044 (local address 127.17.96.254:37363)
0619 02:33:46.196550 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:46.196572 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:46.196611 (+ 39us) server_negotiation.cc:407] Connection header received
0619 02:33:46.197312 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:46.197399 (+ 87us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:46.197996 (+ 597us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:46.198391 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:46.198939 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.200019 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.201139 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.201909 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.202118 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:46.203034 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:46.203042 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:46.203062 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:46.203706 (+ 644us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:46.203788 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:46.203818 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:46.203938 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:46.204306 (+ 368us) server_negotiation.cc:299] Negotiation successful
0619 02:33:46.204726 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":144,"threads_started":1}
I20260619 02:33:46.205837 5091 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b6202ba4df477fae8021cb6d306149" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 37363 } }
I20260619 02:33:46.206197 5091 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:46.206269 5091 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6b6202ba4df477fae8021cb6d306149, State: Initialized, Role: FOLLOWER
I20260619 02:33:46.206808 5091 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b6202ba4df477fae8021cb6d306149" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 37363 } }
I20260619 02:33:46.206982 5091 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:33:46.207060 5091 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:33:46.207166 5091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:46.208338 5091 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b6202ba4df477fae8021cb6d306149" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 37363 } }
I20260619 02:33:46.208787 5091 leader_election.cc:302] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6b6202ba4df477fae8021cb6d306149; no voters:
I20260619 02:33:46.209146 5091 leader_election.cc:288] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:33:46.209232 5096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:33:46.209475 5096 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149 [term 1 LEADER]: Becoming Leader. State: Replica: b6b6202ba4df477fae8021cb6d306149, State: Running, Role: LEADER
I20260619 02:33:46.209851 5096 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b6202ba4df477fae8021cb6d306149" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 37363 } }
I20260619 02:33:46.210696 5091 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:33:46.211860 5097 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6b6202ba4df477fae8021cb6d306149" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b6202ba4df477fae8021cb6d306149" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 37363 } } }
I20260619 02:33:46.211910 5098 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6b6202ba4df477fae8021cb6d306149. Latest consensus state: current_term: 1 leader_uuid: "b6b6202ba4df477fae8021cb6d306149" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6b6202ba4df477fae8021cb6d306149" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 37363 } } }
I20260619 02:33:46.212322 5097 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149 [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:46.212363 5098 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149 [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:46.213442 5106 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:33:46.216678 5106 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:33:46.223913 5106 catalog_manager.cc:1347] Generated new cluster ID: 21e0e410a3b84d9c9a568f8200afaf16
I20260619 02:33:46.224188 5106 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:33:46.234076 5106 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:33:46.235203 5106 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:33:46.244050 5106 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149: Generated new TSK 0
I20260619 02:33:46.244973 5106 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:33:46.255370 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.193:0
--local_ip_for_outbound_sockets=127.17.96.193
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:37363
--builtin_ntp_servers=127.17.96.212:45553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:33:46.387873 5115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:46.388198 5115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:46.388242 5115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:46.392938 5115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:46.393014 5115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:46.393075 5115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.193
I20260619 02:33:46.398226 5115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:45553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.96.193
--webserver_port=0
--tserver_master_addrs=127.17.96.254:37363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.193
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:46.399504 5115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:46.401068 5115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:46.412923 5121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:46.412966 5120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:46.413007 5115 server_base.cc:1034] running on GCE node
W20260619 02:33:46.413192 5123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:46.413697 5115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:46.414474 5115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:46.415658 5115 hybrid_clock.cc:648] HybridClock initialized: now 1781836426415642 us; error 30 us; skew 500 ppm
I20260619 02:33:46.418248 5115 webserver.cc:460] Webserver started at http://127.17.96.193:36517/ using document root <none> and password file <none>
I20260619 02:33:46.418898 5115 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:46.418977 5115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:46.419164 5115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:46.421443 5115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-0/data/instance:
uuid: "9ef9651ed726498da2a1d61f3755f781"
format_stamp: "Formatted at 2026-06-19 02:33:46 on dist-test-slave-4qw6"
server_key: "036e3a8f4a7fdc0a65a55b5583e25561"
server_key_iv: "aa5a4be43bc27a54a641a7b4d006e22f"
server_key_version: "encryptionkey@0"
I20260619 02:33:46.422147 5115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance:
uuid: "9ef9651ed726498da2a1d61f3755f781"
format_stamp: "Formatted at 2026-06-19 02:33:46 on dist-test-slave-4qw6"
server_key: "036e3a8f4a7fdc0a65a55b5583e25561"
server_key_iv: "aa5a4be43bc27a54a641a7b4d006e22f"
server_key_version: "encryptionkey@0"
I20260619 02:33:46.427115 5115 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260619 02:33:46.430368 5129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:46.431895 5115 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:33:46.432024 5115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-0/wal
uuid: "9ef9651ed726498da2a1d61f3755f781"
format_stamp: "Formatted at 2026-06-19 02:33:46 on dist-test-slave-4qw6"
server_key: "036e3a8f4a7fdc0a65a55b5583e25561"
server_key_iv: "aa5a4be43bc27a54a641a7b4d006e22f"
server_key_version: "encryptionkey@0"
I20260619 02:33:46.432145 5115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:46.450366 5115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:46.451305 5115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:46.451478 5115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:46.452124 5115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:33:46.453131 5115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:33:46.453178 5115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:46.453233 5115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:33:46.453255 5115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:46.462720 5115 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.193:38863
I20260619 02:33:46.462750 5242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.193:38863 every 8 connection(s)
I20260619 02:33:46.463734 5115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-0/data/info.pb
I20260619 02:33:46.473448 5094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:46.465860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.193:37385 (local address 127.17.96.254:37363)
0619 02:33:46.466015 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:46.466019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:46.467106 (+ 1087us) server_negotiation.cc:407] Connection header received
0619 02:33:46.468553 (+ 1447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:46.468555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:46.468608 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:46.468690 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:46.469642 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.470298 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.471497 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.471711 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.471798 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:46.472934 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:46.472937 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:46.472939 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:46.472954 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:46.472973 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:46.472981 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:46.473042 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:46.473226 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 02:33:46.473293 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 02:33:46.473948 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 5115
I20260619 02:33:46.474117 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-0/wal/instance
I20260619 02:33:46.474440 17795 external_mini_cluster.cc:1428] Setting key 294410a56055f6204f8f717fa9c87f4b
I20260619 02:33:46.474567 5245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:46.466248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:37363 (local address 127.17.96.193:37385)
0619 02:33:46.466961 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:46.467006 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:46.468308 (+ 1302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:46.468880 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:46.468887 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:46.468976 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:33:46.469494 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:46.469506 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.470433 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.470436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:46.471374 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:46.471384 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.471871 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.471874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:46.472025 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:46.472625 (+ 600us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:33:46.472627 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:33:46.472690 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:33:46.472763 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:33:46.473059 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:33:46.473066 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:33:46.473070 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:33:46.473185 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 02:33:46.473427 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":136,"threads_started":1}
I20260619 02:33:46.475992 5243 heartbeater.cc:347] Connected to a master server at 127.17.96.254:37363
I20260619 02:33:46.476275 5243 heartbeater.cc:464] Registering TS with master...
I20260619 02:33:46.476645 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.194:0
--local_ip_for_outbound_sockets=127.17.96.194
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:37363
--builtin_ntp_servers=127.17.96.212:45553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 02:33:46.476876 5243 heartbeater.cc:511] Master 127.17.96.254:37363 requested a full tablet report, sending...
I20260619 02:33:46.478721 5055 ts_manager.cc:194] Registered new tserver with Master: 9ef9651ed726498da2a1d61f3755f781 (127.17.96.193:38863)
I20260619 02:33:46.480474 5055 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.96.193:37385
W20260619 02:33:46.603427 5246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:46.603926 5246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:46.604013 5246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:46.609459 5246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:46.609550 5246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:46.609620 5246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.194
I20260619 02:33:46.615701 5246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:45553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.96.194
--webserver_port=0
--tserver_master_addrs=127.17.96.254:37363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.194
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:46.616952 5246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:46.618645 5246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:46.630080 5251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:46.630080 5252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:46.630100 5254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:46.630679 5246 server_base.cc:1034] running on GCE node
I20260619 02:33:46.631192 5246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:46.632023 5246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:46.633249 5246 hybrid_clock.cc:648] HybridClock initialized: now 1781836426633228 us; error 40 us; skew 500 ppm
I20260619 02:33:46.635939 5246 webserver.cc:460] Webserver started at http://127.17.96.194:36889/ using document root <none> and password file <none>
I20260619 02:33:46.636683 5246 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:46.636813 5246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:46.637108 5246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:46.639662 5246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-1/data/instance:
uuid: "b1549273365847a1a1ae987033fce549"
format_stamp: "Formatted at 2026-06-19 02:33:46 on dist-test-slave-4qw6"
server_key: "219d0a1f04760c4a02ba5a85c77f05a3"
server_key_iv: "b98a6ec1b9176756fd9591217d5abf1c"
server_key_version: "encryptionkey@0"
I20260619 02:33:46.640353 5246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance:
uuid: "b1549273365847a1a1ae987033fce549"
format_stamp: "Formatted at 2026-06-19 02:33:46 on dist-test-slave-4qw6"
server_key: "219d0a1f04760c4a02ba5a85c77f05a3"
server_key_iv: "b98a6ec1b9176756fd9591217d5abf1c"
server_key_version: "encryptionkey@0"
I20260619 02:33:46.644826 5246 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260619 02:33:46.647701 5260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:46.649102 5246 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:33:46.649267 5246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-1/wal
uuid: "b1549273365847a1a1ae987033fce549"
format_stamp: "Formatted at 2026-06-19 02:33:46 on dist-test-slave-4qw6"
server_key: "219d0a1f04760c4a02ba5a85c77f05a3"
server_key_iv: "b98a6ec1b9176756fd9591217d5abf1c"
server_key_version: "encryptionkey@0"
I20260619 02:33:46.649408 5246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:46.664984 5246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:46.665793 5246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:46.666003 5246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:46.666635 5246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:33:46.667649 5246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:33:46.667721 5246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:46.667806 5246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:33:46.667863 5246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:46.677541 5246 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.194:44013
I20260619 02:33:46.677634 5373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.194:44013 every 8 connection(s)
I20260619 02:33:46.678628 5246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-1/data/info.pb
I20260619 02:33:46.685427 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 5246
I20260619 02:33:46.685577 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-1/wal/instance
I20260619 02:33:46.685964 17795 external_mini_cluster.cc:1428] Setting key 0bb720352e5c2660289070afed552f89
I20260619 02:33:46.688436 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.96.195:0
--local_ip_for_outbound_sockets=127.17.96.195
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:37363
--builtin_ntp_servers=127.17.96.212:45553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 02:33:46.689224 5094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:46.680667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:60885 (local address 127.17.96.254:37363)
0619 02:33:46.680863 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:46.680868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:46.682137 (+ 1269us) server_negotiation.cc:407] Connection header received
0619 02:33:46.683579 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:46.683581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:46.683635 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:46.683720 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:46.684713 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.685295 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.686936 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.687220 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.687307 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:46.688658 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:46.688665 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:46.688668 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:46.688683 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:46.688734 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:46.688747 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:46.688820 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:46.689039 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 02:33:46.689098 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":82}
I20260619 02:33:46.690491 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:46.681155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:37363 (local address 127.17.96.194:60885)
0619 02:33:46.681957 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:46.682005 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:46.683355 (+ 1350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:46.683916 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:46.683924 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:46.684017 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:33:46.684565 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:46.684578 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.685440 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.685445 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:46.686795 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:46.686813 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.687388 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.687400 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:46.687559 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:46.688270 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:33:46.688275 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:33:46.688338 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:33:46.688455 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:33:46.688855 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:33:46.688868 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:33:46.688878 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 02:33:46.689034 (+ 156us) client_negotiation.cc:240] Negotiation successful
0619 02:33:46.689393 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":123,"threads_started":1}
I20260619 02:33:46.691857 5374 heartbeater.cc:347] Connected to a master server at 127.17.96.254:37363
I20260619 02:33:46.692144 5374 heartbeater.cc:464] Registering TS with master...
I20260619 02:33:46.692876 5374 heartbeater.cc:511] Master 127.17.96.254:37363 requested a full tablet report, sending...
I20260619 02:33:46.694195 5055 ts_manager.cc:194] Registered new tserver with Master: b1549273365847a1a1ae987033fce549 (127.17.96.194:44013)
I20260619 02:33:46.694811 5055 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.96.194:60885
W20260619 02:33:46.806967 5377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:46.807309 5377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:46.807353 5377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:46.812302 5377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:46.812388 5377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:46.812455 5377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.96.195
I20260619 02:33:46.818020 5377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:45553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.96.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.96.195
--webserver_port=0
--tserver_master_addrs=127.17.96.254:37363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.96.195
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:46.819389 5377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:46.820987 5377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:46.833089 5383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:46.833014 5382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:46.833014 5385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:46.833197 5377 server_base.cc:1034] running on GCE node
I20260619 02:33:46.834048 5377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:46.834833 5377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:46.836112 5377 hybrid_clock.cc:648] HybridClock initialized: now 1781836426836064 us; error 60 us; skew 500 ppm
I20260619 02:33:46.838495 5377 webserver.cc:460] Webserver started at http://127.17.96.195:35729/ using document root <none> and password file <none>
I20260619 02:33:46.839493 5377 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:46.839690 5377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:46.840013 5377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:46.842859 5377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-2/data/instance:
uuid: "14c822ae5275423e8d67332b5383b845"
format_stamp: "Formatted at 2026-06-19 02:33:46 on dist-test-slave-4qw6"
server_key: "4b30326f445036e2bf6ef7e74cc420c1"
server_key_iv: "397d2ac7c9c5a693c3c00fa825c84132"
server_key_version: "encryptionkey@0"
I20260619 02:33:46.843571 5377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance:
uuid: "14c822ae5275423e8d67332b5383b845"
format_stamp: "Formatted at 2026-06-19 02:33:46 on dist-test-slave-4qw6"
server_key: "4b30326f445036e2bf6ef7e74cc420c1"
server_key_iv: "397d2ac7c9c5a693c3c00fa825c84132"
server_key_version: "encryptionkey@0"
I20260619 02:33:46.848884 5377 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 02:33:46.852134 5391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:46.853698 5377 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 02:33:46.853935 5377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-2/wal
uuid: "14c822ae5275423e8d67332b5383b845"
format_stamp: "Formatted at 2026-06-19 02:33:46 on dist-test-slave-4qw6"
server_key: "4b30326f445036e2bf6ef7e74cc420c1"
server_key_iv: "397d2ac7c9c5a693c3c00fa825c84132"
server_key_version: "encryptionkey@0"
I20260619 02:33:46.854094 5377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:46.873440 5377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:46.874413 5377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:46.874672 5377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:46.875384 5377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:33:46.877106 5377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:33:46.877292 5377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:46.877583 5377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:33:46.877645 5377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:46.891505 5377 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.195:34135
I20260619 02:33:46.891636 5504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.195:34135 every 8 connection(s)
I20260619 02:33:46.892859 5377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-2/data/info.pb
I20260619 02:33:46.897053 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 5377
I20260619 02:33:46.897231 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-2/wal/instance
I20260619 02:33:46.897708 17795 external_mini_cluster.cc:1428] Setting key 611a18456e7a1cc89544ddcd66ee0aeb
I20260619 02:33:46.906741 5094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:46.895995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.195:47747 (local address 127.17.96.254:37363)
0619 02:33:46.896273 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:46.896277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:46.897838 (+ 1561us) server_negotiation.cc:407] Connection header received
0619 02:33:46.900502 (+ 2664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:46.900506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:46.900696 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:46.900870 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:46.902259 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.902859 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.904231 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.904483 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.904615 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:46.906156 (+ 1541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:46.906160 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:46.906162 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:46.906178 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:46.906207 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:46.906216 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:46.906303 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:46.906493 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 02:33:46.906569 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260619 02:33:46.907855 5507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:46.896525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:37363 (local address 127.17.96.195:47747)
0619 02:33:46.897537 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:46.897611 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:46.900176 (+ 2565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:46.901161 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:46.901173 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:46.901309 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:33:46.902061 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:46.902082 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.903041 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.903044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:46.904062 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:46.904074 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.904673 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.904677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:46.904875 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:46.905818 (+ 943us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:33:46.905820 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:33:46.905872 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:33:46.905960 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:33:46.906318 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:33:46.906331 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:33:46.906343 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 02:33:46.906482 (+ 139us) client_negotiation.cc:240] Negotiation successful
0619 02:33:46.906861 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":189,"threads_started":1}
I20260619 02:33:46.909384 5505 heartbeater.cc:347] Connected to a master server at 127.17.96.254:37363
I20260619 02:33:46.909925 5505 heartbeater.cc:464] Registering TS with master...
I20260619 02:33:46.910872 5505 heartbeater.cc:511] Master 127.17.96.254:37363 requested a full tablet report, sending...
I20260619 02:33:46.912524 5049 ts_manager.cc:194] Registered new tserver with Master: 14c822ae5275423e8d67332b5383b845 (127.17.96.195:34135)
I20260619 02:33:46.913472 5049 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.96.195:47747
I20260619 02:33:46.913690 17795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:33:46.920967 5094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:46.916453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41046 (local address 127.17.96.254:37363)
0619 02:33:46.916639 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:46.916645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:46.916822 (+ 177us) server_negotiation.cc:407] Connection header received
0619 02:33:46.917024 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:46.917028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:46.917089 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:46.917183 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:46.917517 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.918205 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.919167 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.919434 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.919560 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:46.920488 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:46.920492 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:46.920494 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:46.920508 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:46.920527 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:46.920535 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:46.920622 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:46.920723 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 02:33:46.920789 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 02:33:46.925237 5049 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41046:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:33:46.928669 5049 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:33:46.947240 5516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:46.940260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.194:44013 (local address 127.0.0.1:46102)
0619 02:33:46.940829 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:46.940844 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:46.940948 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:46.941853 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:46.941856 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:46.941878 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:46.941992 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:46.941999 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.943554 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.943559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:46.945284 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:46.945296 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.946703 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.946707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:46.946820 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:46.946838 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 02:33:46.946909 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 02:33:46.947029 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":132,"thread_start_us":118,"threads_started":1}
I20260619 02:33:46.947257 5521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:46.940934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46102 (local address 127.17.96.194:44013)
0619 02:33:46.941343 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:46.941356 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:46.941381 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:33:46.941472 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:46.941480 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:46.941684 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:46.941890 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:46.942166 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.943444 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.945866 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.946517 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.946681 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:46.946887 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:46.946949 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 02:33:46.947073 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":172,"threads_started":1}
I20260619 02:33:46.947458 5518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:46.939345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40334 (local address 127.17.96.195:34135)
0619 02:33:46.940632 (+ 1287us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:46.940646 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:46.940671 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:33:46.940868 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:46.940880 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:46.941094 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:46.941319 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:46.941673 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.943118 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.945437 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.946303 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.946417 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:46.946577 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:46.947124 (+ 547us) server_negotiation.cc:299] Negotiation successful
0619 02:33:46.947239 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1169,"thread_start_us":188,"threads_started":1}
I20260619 02:33:46.948037 5520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:46.939248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:34135 (local address 127.0.0.1:40334)
0619 02:33:46.940536 (+ 1288us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:46.940552 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:46.940657 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:46.941308 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:46.941315 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:46.941340 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:46.941511 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:46.941521 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.943272 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.943276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:46.945284 (+ 2008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:46.945296 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.946449 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.946455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:46.946598 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:46.946665 (+ 67us) client_negotiation.cc:769] Sending connection context
0619 02:33:46.946788 (+ 123us) client_negotiation.cc:240] Negotiation successful
0619 02:33:46.946965 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1212,"mutex_wait_us":201,"thread_start_us":104,"threads_started":1}
I20260619 02:33:46.948035 5519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:46.939677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56264 (local address 127.17.96.193:38863)
0619 02:33:46.940330 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:46.940343 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:46.940395 (+ 52us) server_negotiation.cc:407] Connection header received
0619 02:33:46.941644 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:46.941650 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:46.941886 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:46.942045 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:46.942387 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.943330 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.945182 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.946132 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.946277 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:46.946567 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:46.947682 (+ 1115us) server_negotiation.cc:299] Negotiation successful
0619 02:33:46.947845 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":110,"threads_started":1}
I20260619 02:33:46.947240 5517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:46.939238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:38863 (local address 127.0.0.1:56264)
0619 02:33:46.940244 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:46.940308 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:46.940467 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:46.942097 (+ 1630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:46.942100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:46.942120 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:46.942242 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:46.942249 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.943489 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.943496 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:46.945032 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:46.945042 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:46.946646 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:46.946652 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:46.946767 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:46.946830 (+ 63us) client_negotiation.cc:769] Sending connection context
0619 02:33:46.946911 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 02:33:46.947029 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"mutex_wait_us":101,"thread_start_us":134,"threads_started":1}
I20260619 02:33:46.951570 5308 tablet_service.cc:1467] Processing CreateTablet for tablet 6a1e41a9ee82424691fee8c83cc61231 (DEFAULT_TABLE table=test-table [id=ff26b17ba7e1470dbf66ff783efb8de8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:33:46.951570 5439 tablet_service.cc:1467] Processing CreateTablet for tablet 6a1e41a9ee82424691fee8c83cc61231 (DEFAULT_TABLE table=test-table [id=ff26b17ba7e1470dbf66ff783efb8de8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:33:46.951586 5177 tablet_service.cc:1467] Processing CreateTablet for tablet 6a1e41a9ee82424691fee8c83cc61231 (DEFAULT_TABLE table=test-table [id=ff26b17ba7e1470dbf66ff783efb8de8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:33:46.952848 5177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a1e41a9ee82424691fee8c83cc61231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:46.952844 5308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a1e41a9ee82424691fee8c83cc61231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:46.952844 5439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a1e41a9ee82424691fee8c83cc61231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:46.962460 5522 tablet_bootstrap.cc:492] T 6a1e41a9ee82424691fee8c83cc61231 P 9ef9651ed726498da2a1d61f3755f781: Bootstrap starting.
I20260619 02:33:46.964421 5523 tablet_bootstrap.cc:492] T 6a1e41a9ee82424691fee8c83cc61231 P 14c822ae5275423e8d67332b5383b845: Bootstrap starting.
I20260619 02:33:46.965687 5522 tablet_bootstrap.cc:654] T 6a1e41a9ee82424691fee8c83cc61231 P 9ef9651ed726498da2a1d61f3755f781: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:46.966063 5524 tablet_bootstrap.cc:492] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549: Bootstrap starting.
I20260619 02:33:46.966879 5522 log.cc:824] T 6a1e41a9ee82424691fee8c83cc61231 P 9ef9651ed726498da2a1d61f3755f781: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:46.969672 5523 tablet_bootstrap.cc:654] T 6a1e41a9ee82424691fee8c83cc61231 P 14c822ae5275423e8d67332b5383b845: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:46.970301 5524 tablet_bootstrap.cc:654] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:46.970515 5522 tablet_bootstrap.cc:492] T 6a1e41a9ee82424691fee8c83cc61231 P 9ef9651ed726498da2a1d61f3755f781: No bootstrap required, opened a new log
I20260619 02:33:46.970796 5522 ts_tablet_manager.cc:1397] T 6a1e41a9ee82424691fee8c83cc61231 P 9ef9651ed726498da2a1d61f3755f781: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20260619 02:33:46.970868 5523 log.cc:824] T 6a1e41a9ee82424691fee8c83cc61231 P 14c822ae5275423e8d67332b5383b845: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:46.971436 5524 log.cc:824] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:46.973831 5523 tablet_bootstrap.cc:492] T 6a1e41a9ee82424691fee8c83cc61231 P 14c822ae5275423e8d67332b5383b845: No bootstrap required, opened a new log
I20260619 02:33:46.974092 5523 ts_tablet_manager.cc:1397] T 6a1e41a9ee82424691fee8c83cc61231 P 14c822ae5275423e8d67332b5383b845: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260619 02:33:46.975971 5524 tablet_bootstrap.cc:492] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549: No bootstrap required, opened a new log
I20260619 02:33:46.976260 5524 ts_tablet_manager.cc:1397] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260619 02:33:46.976222 5522 raft_consensus.cc:348] T 6a1e41a9ee82424691fee8c83cc61231 P 9ef9651ed726498da2a1d61f3755f781 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14c822ae5275423e8d67332b5383b845" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 34135 } } peers { permanent_uuid: "9ef9651ed726498da2a1d61f3755f781" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 38863 } } peers { permanent_uuid: "b1549273365847a1a1ae987033fce549" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44013 } }
I20260619 02:33:46.976620 5522 raft_consensus.cc:374] T 6a1e41a9ee82424691fee8c83cc61231 P 9ef9651ed726498da2a1d61f3755f781 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:46.976691 5522 raft_consensus.cc:729] T 6a1e41a9ee82424691fee8c83cc61231 P 9ef9651ed726498da2a1d61f3755f781 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ef9651ed726498da2a1d61f3755f781, State: Initialized, Role: FOLLOWER
I20260619 02:33:46.977311 5522 consensus_queue.cc:260] T 6a1e41a9ee82424691fee8c83cc61231 P 9ef9651ed726498da2a1d61f3755f781 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14c822ae5275423e8d67332b5383b845" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 34135 } } peers { permanent_uuid: "9ef9651ed726498da2a1d61f3755f781" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 38863 } } peers { permanent_uuid: "b1549273365847a1a1ae987033fce549" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44013 } }
I20260619 02:33:46.978336 5243 heartbeater.cc:503] Master 127.17.96.254:37363 was elected leader, sending a full tablet report...
I20260619 02:33:46.978814 5522 ts_tablet_manager.cc:1428] T 6a1e41a9ee82424691fee8c83cc61231 P 9ef9651ed726498da2a1d61f3755f781: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260619 02:33:46.979156 5523 raft_consensus.cc:348] T 6a1e41a9ee82424691fee8c83cc61231 P 14c822ae5275423e8d67332b5383b845 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14c822ae5275423e8d67332b5383b845" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 34135 } } peers { permanent_uuid: "9ef9651ed726498da2a1d61f3755f781" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 38863 } } peers { permanent_uuid: "b1549273365847a1a1ae987033fce549" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44013 } }
I20260619 02:33:46.979522 5523 raft_consensus.cc:374] T 6a1e41a9ee82424691fee8c83cc61231 P 14c822ae5275423e8d67332b5383b845 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:46.979588 5523 raft_consensus.cc:729] T 6a1e41a9ee82424691fee8c83cc61231 P 14c822ae5275423e8d67332b5383b845 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14c822ae5275423e8d67332b5383b845, State: Initialized, Role: FOLLOWER
I20260619 02:33:46.980177 5523 consensus_queue.cc:260] T 6a1e41a9ee82424691fee8c83cc61231 P 14c822ae5275423e8d67332b5383b845 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14c822ae5275423e8d67332b5383b845" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 34135 } } peers { permanent_uuid: "9ef9651ed726498da2a1d61f3755f781" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 38863 } } peers { permanent_uuid: "b1549273365847a1a1ae987033fce549" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44013 } }
I20260619 02:33:46.981037 5505 heartbeater.cc:503] Master 127.17.96.254:37363 was elected leader, sending a full tablet report...
I20260619 02:33:46.981525 5524 raft_consensus.cc:348] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14c822ae5275423e8d67332b5383b845" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 34135 } } peers { permanent_uuid: "9ef9651ed726498da2a1d61f3755f781" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 38863 } } peers { permanent_uuid: "b1549273365847a1a1ae987033fce549" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44013 } }
I20260619 02:33:46.981966 5524 raft_consensus.cc:374] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:46.982053 5524 raft_consensus.cc:729] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1549273365847a1a1ae987033fce549, State: Initialized, Role: FOLLOWER
I20260619 02:33:46.982719 5524 consensus_queue.cc:260] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14c822ae5275423e8d67332b5383b845" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 34135 } } peers { permanent_uuid: "9ef9651ed726498da2a1d61f3755f781" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 38863 } } peers { permanent_uuid: "b1549273365847a1a1ae987033fce549" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44013 } }
I20260619 02:33:46.982879 5523 ts_tablet_manager.cc:1428] T 6a1e41a9ee82424691fee8c83cc61231 P 14c822ae5275423e8d67332b5383b845: Time spent starting tablet: real 0.009s user 0.006s sys 0.001s
I20260619 02:33:46.983772 5374 heartbeater.cc:503] Master 127.17.96.254:37363 was elected leader, sending a full tablet report...
I20260619 02:33:46.984222 5524 ts_tablet_manager.cc:1428] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
I20260619 02:33:47.086920 5530 raft_consensus.cc:482] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:33:47.087325 5530 raft_consensus.cc:504] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14c822ae5275423e8d67332b5383b845" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 34135 } } peers { permanent_uuid: "9ef9651ed726498da2a1d61f3755f781" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 38863 } } peers { permanent_uuid: "b1549273365847a1a1ae987033fce549" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44013 } }
I20260619 02:33:47.088994 5530 leader_election.cc:288] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14c822ae5275423e8d67332b5383b845 (127.17.96.195:34135), 9ef9651ed726498da2a1d61f3755f781 (127.17.96.193:38863)
I20260619 02:33:47.093925 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:47.089418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:38863 (local address 127.17.96.194:44161)
0619 02:33:47.089580 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:47.089607 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:47.089754 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:47.090157 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:47.090161 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:47.090198 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:47.090380 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:47.090388 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:47.091307 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:47.091314 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:47.092472 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:47.092485 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:47.093250 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:47.093254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:47.093442 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:47.093467 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 02:33:47.093571 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 02:33:47.093690 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":5}
I20260619 02:33:47.093899 5519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:47.089583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:44161 (local address 127.17.96.193:38863)
0619 02:33:47.089810 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:47.089818 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:47.089840 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:33:47.089915 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:47.089919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:47.089982 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:47.090062 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:47.090570 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:47.091145 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:47.092635 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:47.093093 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:47.093255 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:47.093321 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:47.093594 (+ 273us) server_negotiation.cc:299] Negotiation successful
0619 02:33:47.093689 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 02:33:47.093897 5518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:47.090131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.96.194:47679 (local address 127.17.96.195:34135)
0619 02:33:47.090279 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:47.090283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:47.090295 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:33:47.090351 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:47.090354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:47.090399 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:47.090510 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:33:47.090837 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:47.091384 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:47.092664 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:47.093120 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:47.093263 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:47.093329 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:47.093604 (+ 275us) server_negotiation.cc:299] Negotiation successful
0619 02:33:47.093695 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:33:47.093995 5531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:47.089546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.195:34135 (local address 127.17.96.194:47679)
0619 02:33:47.090065 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:47.090087 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 02:33:47.090212 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:33:47.090532 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:33:47.090536 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:33:47.090555 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:33:47.090722 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:47.090728 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:47.091512 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:47.091518 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:47.092532 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:33:47.092542 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:33:47.093256 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:33:47.093260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:33:47.093472 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:47.093493 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 02:33:47.093586 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 02:33:47.093684 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"mutex_wait_us":159,"spinlock_wait_cycles":9984,"thread_start_us":248,"threads_started":1}
I20260619 02:33:47.094942 5459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a1e41a9ee82424691fee8c83cc61231" candidate_uuid: "b1549273365847a1a1ae987033fce549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14c822ae5275423e8d67332b5383b845" is_pre_election: true
I20260619 02:33:47.094962 5197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a1e41a9ee82424691fee8c83cc61231" candidate_uuid: "b1549273365847a1a1ae987033fce549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ef9651ed726498da2a1d61f3755f781" is_pre_election: true
I20260619 02:33:47.095376 5197 raft_consensus.cc:2454] T 6a1e41a9ee82424691fee8c83cc61231 P 9ef9651ed726498da2a1d61f3755f781 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1549273365847a1a1ae987033fce549 in term 0.
I20260619 02:33:47.095378 5459 raft_consensus.cc:2454] T 6a1e41a9ee82424691fee8c83cc61231 P 14c822ae5275423e8d67332b5383b845 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1549273365847a1a1ae987033fce549 in term 0.
I20260619 02:33:47.096060 5262 leader_election.cc:302] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 14c822ae5275423e8d67332b5383b845, 9ef9651ed726498da2a1d61f3755f781, b1549273365847a1a1ae987033fce549; no voters:
I20260619 02:33:47.096449 5530 raft_consensus.cc:2781] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:33:47.096542 5530 raft_consensus.cc:482] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:33:47.096577 5530 raft_consensus.cc:3037] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:47.098155 5530 raft_consensus.cc:504] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14c822ae5275423e8d67332b5383b845" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 34135 } } peers { permanent_uuid: "9ef9651ed726498da2a1d61f3755f781" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 38863 } } peers { permanent_uuid: "b1549273365847a1a1ae987033fce549" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44013 } }
I20260619 02:33:47.098810 5530 leader_election.cc:288] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549 [CANDIDATE]: Term 1 election: Requested vote from peers 14c822ae5275423e8d67332b5383b845 (127.17.96.195:34135), 9ef9651ed726498da2a1d61f3755f781 (127.17.96.193:38863)
I20260619 02:33:47.099634 5197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a1e41a9ee82424691fee8c83cc61231" candidate_uuid: "b1549273365847a1a1ae987033fce549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ef9651ed726498da2a1d61f3755f781"
I20260619 02:33:47.099634 5459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a1e41a9ee82424691fee8c83cc61231" candidate_uuid: "b1549273365847a1a1ae987033fce549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14c822ae5275423e8d67332b5383b845"
I20260619 02:33:47.099807 5197 raft_consensus.cc:3037] T 6a1e41a9ee82424691fee8c83cc61231 P 9ef9651ed726498da2a1d61f3755f781 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:47.099807 5459 raft_consensus.cc:3037] T 6a1e41a9ee82424691fee8c83cc61231 P 14c822ae5275423e8d67332b5383b845 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:47.101847 5459 raft_consensus.cc:2454] T 6a1e41a9ee82424691fee8c83cc61231 P 14c822ae5275423e8d67332b5383b845 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1549273365847a1a1ae987033fce549 in term 1.
I20260619 02:33:47.101877 5197 raft_consensus.cc:2454] T 6a1e41a9ee82424691fee8c83cc61231 P 9ef9651ed726498da2a1d61f3755f781 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1549273365847a1a1ae987033fce549 in term 1.
I20260619 02:33:47.102620 5261 leader_election.cc:302] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 14c822ae5275423e8d67332b5383b845, 9ef9651ed726498da2a1d61f3755f781, b1549273365847a1a1ae987033fce549; no voters:
I20260619 02:33:47.102932 5530 raft_consensus.cc:2781] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:33:47.103408 5530 raft_consensus.cc:686] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549 [term 1 LEADER]: Becoming Leader. State: Replica: b1549273365847a1a1ae987033fce549, State: Running, Role: LEADER
I20260619 02:33:47.104008 5530 consensus_queue.cc:237] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14c822ae5275423e8d67332b5383b845" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 34135 } } peers { permanent_uuid: "9ef9651ed726498da2a1d61f3755f781" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 38863 } } peers { permanent_uuid: "b1549273365847a1a1ae987033fce549" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44013 } }
I20260619 02:33:47.108822 5055 catalog_manager.cc:5495] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1549273365847a1a1ae987033fce549 (127.17.96.194). New cstate: current_term: 1 leader_uuid: "b1549273365847a1a1ae987033fce549" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14c822ae5275423e8d67332b5383b845" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 34135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ef9651ed726498da2a1d61f3755f781" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 38863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1549273365847a1a1ae987033fce549" member_type: VOTER last_known_addr { host: "127.17.96.194" port: 44013 } health_report { overall_health: HEALTHY } } }
W20260619 02:33:47.144156 5506 tablet.cc:2369] T 6a1e41a9ee82424691fee8c83cc61231 P 14c822ae5275423e8d67332b5383b845: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:33:47.144454 5521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:47.140976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46106 (local address 127.17.96.194:44013)
0619 02:33:47.141111 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:47.141115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:47.141129 (+ 14us) server_negotiation.cc:407] Connection header received
0619 02:33:47.141208 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:47.141211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:47.141268 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:47.141362 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:33:47.141575 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:47.142116 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:47.142929 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:47.143131 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:47.143206 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:47.143492 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:33:47.143969 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:33:47.144113 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:47.144200 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 02:33:47.144257 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 02:33:47.150734 5197 raft_consensus.cc:1261] T 6a1e41a9ee82424691fee8c83cc61231 P 9ef9651ed726498da2a1d61f3755f781 [term 1 FOLLOWER]: Refusing update from remote peer b1549273365847a1a1ae987033fce549: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:33:47.150753 5459 raft_consensus.cc:1261] T 6a1e41a9ee82424691fee8c83cc61231 P 14c822ae5275423e8d67332b5383b845 [term 1 FOLLOWER]: Refusing update from remote peer b1549273365847a1a1ae987033fce549: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:33:47.151607 5532 consensus_queue.cc:1035] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14c822ae5275423e8d67332b5383b845" member_type: VOTER last_known_addr { host: "127.17.96.195" port: 34135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:33:47.151937 5530 consensus_queue.cc:1035] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ef9651ed726498da2a1d61f3755f781" member_type: VOTER last_known_addr { host: "127.17.96.193" port: 38863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:33:47.158830 5535 mvcc.cc:205] Tried to move back new op lower bound from 7298402005601083392 to 7298402005422280704. Current Snapshot: MvccSnapshot[applied={T|T < 7298402005601083392}]
I20260619 02:33:47.160521 5536 mvcc.cc:205] Tried to move back new op lower bound from 7298402005601083392 to 7298402005422280704. Current Snapshot: MvccSnapshot[applied={T|T < 7298402005601083392}]
I20260619 02:33:47.169401 5055 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:33:47.169642 5055 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:33:47.172583 5055 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6b6202ba4df477fae8021cb6d306149: Sending DeleteTablet for 3 replicas of tablet 6a1e41a9ee82424691fee8c83cc61231
I20260619 02:33:47.173588 5439 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a1e41a9ee82424691fee8c83cc61231 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:33:47 UTC) from {username='slave'} at 127.0.0.1:40334
I20260619 02:33:47.173591 5177 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a1e41a9ee82424691fee8c83cc61231 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:33:47 UTC) from {username='slave'} at 127.0.0.1:56264
I20260619 02:33:47.173826 5308 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a1e41a9ee82424691fee8c83cc61231 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:33:47 UTC) from {username='slave'} at 127.0.0.1:46102
I20260619 02:33:47.174293 5544 tablet_replica.cc:330] T 6a1e41a9ee82424691fee8c83cc61231 P 14c822ae5275423e8d67332b5383b845: stopping tablet replica
I20260619 02:33:47.174297 5546 tablet_replica.cc:330] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549: stopping tablet replica
I20260619 02:33:47.174669 5546 raft_consensus.cc:2229] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:33:47.174676 5544 raft_consensus.cc:2229] T 6a1e41a9ee82424691fee8c83cc61231 P 14c822ae5275423e8d67332b5383b845 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:33:47.174777 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 5115
I20260619 02:33:47.175064 5544 raft_consensus.cc:2258] T 6a1e41a9ee82424691fee8c83cc61231 P 14c822ae5275423e8d67332b5383b845 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:33:47.175742 5546 raft_consensus.cc:2258] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:33:47.177728 5544 ts_tablet_manager.cc:1905] T 6a1e41a9ee82424691fee8c83cc61231 P 14c822ae5275423e8d67332b5383b845: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:33:47.179410 5546 ts_tablet_manager.cc:1905] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:33:47.181595 5544 ts_tablet_manager.cc:1918] T 6a1e41a9ee82424691fee8c83cc61231 P 14c822ae5275423e8d67332b5383b845: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:33:47.181722 5544 log.cc:1196] T 6a1e41a9ee82424691fee8c83cc61231 P 14c822ae5275423e8d67332b5383b845: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-2/wal/wals/6a1e41a9ee82424691fee8c83cc61231
I20260619 02:33:47.182089 5544 ts_tablet_manager.cc:1939] T 6a1e41a9ee82424691fee8c83cc61231 P 14c822ae5275423e8d67332b5383b845: Deleting consensus metadata
I20260619 02:33:47.183403 5546 ts_tablet_manager.cc:1918] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:33:47.183676 5546 log.cc:1196] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549: Deleting WAL directory at /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781836162001484-17795-0/minicluster-data/ts-1/wal/wals/6a1e41a9ee82424691fee8c83cc61231
I20260619 02:33:47.183753 5039 catalog_manager.cc:4841] TS 14c822ae5275423e8d67332b5383b845 (127.17.96.195:34135): tablet 6a1e41a9ee82424691fee8c83cc61231 (table test-table [id=ff26b17ba7e1470dbf66ff783efb8de8]) successfully deleted
I20260619 02:33:47.184142 5546 ts_tablet_manager.cc:1939] T 6a1e41a9ee82424691fee8c83cc61231 P b1549273365847a1a1ae987033fce549: Deleting consensus metadata
I20260619 02:33:47.185418 5042 catalog_manager.cc:4841] TS b1549273365847a1a1ae987033fce549 (127.17.96.194:44013): tablet 6a1e41a9ee82424691fee8c83cc61231 (table test-table [id=ff26b17ba7e1470dbf66ff783efb8de8]) successfully deleted
W20260619 02:33:47.185918 5040 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.96.193:38863 (error 108)
I20260619 02:33:47.186192 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 5246
I20260619 02:33:47.187810 5520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:47.187411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.193:38863 (local address 127.0.0.1:56280)
0619 02:33:47.187625 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 02:33:47.187703 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.193:38863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260619 02:33:47.187991 5040 catalog_manager.cc:4584] TS 9ef9651ed726498da2a1d61f3755f781 (127.17.96.193:38863): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6a1e41a9ee82424691fee8c83cc61231: Network error: Client connection negotiation failed: client connection to 127.17.96.193:38863: connect: Connection refused (error 111)
I20260619 02:33:47.192770 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 5377
I20260619 02:33:47.199188 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 5024
2026-06-19T02:33:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1254 ms)
[----------] 6 tests from EncryptionPolicyTest (11184 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:33:47 dist-test-slave-4qw6 krb5kdc[5550](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:33:47 dist-test-slave-4qw6 krb5kdc[5550](info): set up 2 sockets
Jun 19 02:33:47 dist-test-slave-4qw6 krb5kdc[5550](info): commencing operation
krb5kdc: starting...
W20260619 02:33:48.318552 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:33:48 dist-test-slave-4qw6 krb5kdc[5550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:33:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:33:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:33:48.457417 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37415
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:33:48.582005 5563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:48.582412 5563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:48.582499 5563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:48.587350 5563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:33:48.587483 5563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:48.587527 5563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:48.587570 5563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:33:48.587610 5563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:33:48.593863 5563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37415
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:48.595445 5563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:48.596601 5563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:33:48.608489 5563 server_base.cc:1034] running on GCE node
W20260619 02:33:48.608382 5569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:48.608381 5571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:48.608347 5568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:48.609500 5563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:48.610903 5563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:48.612114 5563 hybrid_clock.cc:648] HybridClock initialized: now 1781836428612094 us; error 41 us; skew 500 ppm
Jun 19 02:33:48 dist-test-slave-4qw6 krb5kdc[5550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836428, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:48.623482 5563 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:33:48.625155 5563 webserver.cc:460] Webserver started at http://127.0.0.1:38265/ using document root <none> and password file <none>
I20260619 02:33:48.625998 5563 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:48.626087 5563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:48.626374 5563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:48.628881 5563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "b731d573235147208101912d26a69946"
format_stamp: "Formatted at 2026-06-19 02:33:48 on dist-test-slave-4qw6"
server_key: "e33e30ae93369d6fd045f6102f40243e"
server_key_iv: "949f8f38f26b670f49d554b94ea582db"
server_key_version: "encryptionkey@0"
I20260619 02:33:48.629616 5563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "b731d573235147208101912d26a69946"
format_stamp: "Formatted at 2026-06-19 02:33:48 on dist-test-slave-4qw6"
server_key: "e33e30ae93369d6fd045f6102f40243e"
server_key_iv: "949f8f38f26b670f49d554b94ea582db"
server_key_version: "encryptionkey@0"
I20260619 02:33:48.634410 5563 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 02:33:48.637621 5578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:48.639549 5563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:33:48.639775 5563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "b731d573235147208101912d26a69946"
format_stamp: "Formatted at 2026-06-19 02:33:48 on dist-test-slave-4qw6"
server_key: "e33e30ae93369d6fd045f6102f40243e"
server_key_iv: "949f8f38f26b670f49d554b94ea582db"
server_key_version: "encryptionkey@0"
I20260619 02:33:48.639955 5563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:48.662447 5563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:48.666128 5563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:48.666399 5563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:48.674399 5563 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37415
I20260619 02:33:48.674423 5630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37415 every 8 connection(s)
I20260619 02:33:48.675594 5563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:33:48.676787 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 5563
I20260619 02:33:48.677021 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:33:48.677414 17795 external_mini_cluster.cc:1428] Setting key c9141a84b91cb745fa6fdc3a056a0e14
I20260619 02:33:48.680178 5631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:48.687225 5631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946: Bootstrap starting.
Jun 19 02:33:48 dist-test-slave-4qw6 krb5kdc[5550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 02:33:48.690335 5631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:48.691171 5631 log.cc:824] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:48.693411 5631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946: No bootstrap required, opened a new log
I20260619 02:33:48.694800 5634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:48.679358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37123 (local address 127.0.0.1:37415)
0619 02:33:48.679984 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:48.680003 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:48.680048 (+ 45us) server_negotiation.cc:407] Connection header received
0619 02:33:48.680897 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:48.680919 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:48.681368 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:48.681819 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:48.682836 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:48.683985 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:48.685027 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:48.685574 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:48.685801 (+ 227us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:48.689134 (+ 3333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:48.689140 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:48.689156 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:33:48.689191 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:48.691192 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:48.691867 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:48.691873 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:48.691878 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:48.692024 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:48.692416 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:48.692420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:48.692423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:48.692875 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:48.693134 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:48.693485 (+ 351us) server_negotiation.cc:299] Negotiation successful
0619 02:33:48.693980 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":206,"threads_started":1}
I20260619 02:33:48.697904 5631 raft_consensus.cc:348] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b731d573235147208101912d26a69946" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37415 } }
I20260619 02:33:48.698230 5631 raft_consensus.cc:374] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:48.698329 5631 raft_consensus.cc:729] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b731d573235147208101912d26a69946, State: Initialized, Role: FOLLOWER
I20260619 02:33:48.698671 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:33:48.698935 5631 consensus_queue.cc:260] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b731d573235147208101912d26a69946" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37415 } }
I20260619 02:33:48.699120 5631 raft_consensus.cc:388] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:33:48.699220 5631 raft_consensus.cc:482] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:33:48.699344 5631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:48.700665 5631 raft_consensus.cc:504] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b731d573235147208101912d26a69946" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37415 } }
I20260619 02:33:48.701216 5631 leader_election.cc:302] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b731d573235147208101912d26a69946; no voters:
I20260619 02:33:48.701670 5631 leader_election.cc:288] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:33:48.701838 5641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:33:48.702083 5641 raft_consensus.cc:686] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946 [term 1 LEADER]: Becoming Leader. State: Replica: b731d573235147208101912d26a69946, State: Running, Role: LEADER
I20260619 02:33:48.702517 5641 consensus_queue.cc:237] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b731d573235147208101912d26a69946" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37415 } }
I20260619 02:33:48.703164 5631 sys_catalog.cc:564] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:33:48.704066 5643 sys_catalog.cc:455] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b731d573235147208101912d26a69946" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b731d573235147208101912d26a69946" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37415 } } }
I20260619 02:33:48.704066 5641 sys_catalog.cc:455] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b731d573235147208101912d26a69946. Latest consensus state: current_term: 1 leader_uuid: "b731d573235147208101912d26a69946" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b731d573235147208101912d26a69946" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37415 } } }
I20260619 02:33:48.704238 5643 sys_catalog.cc:458] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946 [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:48.704239 5641 sys_catalog.cc:458] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946 [sys.catalog]: This master's current role is: LEADER
W20260619 02:33:48.709323 5656 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:33:48.709458 5656 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:33:48.709549 5650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:33:48.710603 5650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:33:48.711861 5634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:48.701258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45813 (local address 127.0.0.1:37415)
0619 02:33:48.701497 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:48.701503 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:48.701587 (+ 84us) server_negotiation.cc:407] Connection header received
0619 02:33:48.701878 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:48.701882 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:48.701941 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:48.702051 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:48.703216 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:48.704095 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:48.705530 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:48.706133 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:48.706255 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:48.707903 (+ 1648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:48.707907 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:48.707910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:33:48.707948 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:48.709841 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:48.710403 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:48.710407 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:48.710409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:48.710502 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:48.710889 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:48.710893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:48.710895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:48.711098 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:48.711261 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:48.711574 (+ 313us) server_negotiation.cc:299] Negotiation successful
0619 02:33:48.711678 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 02:33:48.718261 5650 catalog_manager.cc:1347] Generated new cluster ID: 621c11bf88f64e5a82baee7b0454a6ac
I20260619 02:33:48.718416 5650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:33:48.733336 5650 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:33:48.734609 5650 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:33:48.741638 5650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b731d573235147208101912d26a69946: Generated new TSK 0
I20260619 02:33:48.742671 5650 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:33:48.752604 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 5563
W20260619 02:33:48.758688 5548 connection.cc:535] client connection to 127.0.0.1:37415 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37415): Connection reset by peer (error 104)
2026-06-19T02:33:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1552 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:33:48 dist-test-slave-4qw6 krb5kdc[5662](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:33:48 dist-test-slave-4qw6 krb5kdc[5662](info): set up 2 sockets
Jun 19 02:33:48 dist-test-slave-4qw6 krb5kdc[5662](info): commencing operation
krb5kdc: starting...
W20260619 02:33:49.894366 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:33:49 dist-test-slave-4qw6 krb5kdc[5662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:33:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:33:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:33:50.042510 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38185
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:33:50.176882 5675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:50.177253 5675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:50.177304 5675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:50.181931 5675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:33:50.182019 5675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:50.182042 5675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:50.182062 5675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:33:50.182080 5675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:33:50.188321 5675 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38185
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:50.189651 5675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:50.190928 5675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:50.201668 5683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:50.201668 5681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:50.201680 5680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:50.202172 5675 server_base.cc:1034] running on GCE node
I20260619 02:33:50.203517 5675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:50.205412 5675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:50.206696 5675 hybrid_clock.cc:648] HybridClock initialized: now 1781836430206660 us; error 88 us; skew 500 ppm
Jun 19 02:33:50 dist-test-slave-4qw6 krb5kdc[5662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836430, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:50.210887 5675 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:33:50.212208 5675 webserver.cc:460] Webserver started at http://127.0.0.1:37519/ using document root <none> and password file <none>
I20260619 02:33:50.212935 5675 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:50.213052 5675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:50.213339 5675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:50.215873 5675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "c7224b292e534cec9ae175ceca965be1"
format_stamp: "Formatted at 2026-06-19 02:33:50 on dist-test-slave-4qw6"
server_key: "a3807413fb53469bb5eeb66999795686"
server_key_iv: "8f322f5dcb8945d58ffb460b240ea4c6"
server_key_version: "encryptionkey@0"
I20260619 02:33:50.216856 5675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "c7224b292e534cec9ae175ceca965be1"
format_stamp: "Formatted at 2026-06-19 02:33:50 on dist-test-slave-4qw6"
server_key: "a3807413fb53469bb5eeb66999795686"
server_key_iv: "8f322f5dcb8945d58ffb460b240ea4c6"
server_key_version: "encryptionkey@0"
I20260619 02:33:50.222012 5675 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 02:33:50.225277 5690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:50.227205 5675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:33:50.227361 5675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "c7224b292e534cec9ae175ceca965be1"
format_stamp: "Formatted at 2026-06-19 02:33:50 on dist-test-slave-4qw6"
server_key: "a3807413fb53469bb5eeb66999795686"
server_key_iv: "8f322f5dcb8945d58ffb460b240ea4c6"
server_key_version: "encryptionkey@0"
I20260619 02:33:50.227571 5675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:50.241670 5675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:50.245883 5675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:50.246129 5675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:50.255034 5675 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38185
I20260619 02:33:50.255062 5742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38185 every 8 connection(s)
I20260619 02:33:50.256248 5675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:33:50.260505 5743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:50.261994 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 5675
I20260619 02:33:50.262161 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:33:50.262555 17795 external_mini_cluster.cc:1428] Setting key 89aa5e39d1796cb19fc49c43b3537cac
I20260619 02:33:50.268296 5743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1: Bootstrap starting.
I20260619 02:33:50.271099 5743 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:50.271976 5743 log.cc:824] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:50.274438 5746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:50.264849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33895 (local address 127.0.0.1:38185)
0619 02:33:50.265498 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:50.265517 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:50.265567 (+ 50us) server_negotiation.cc:407] Connection header received
0619 02:33:50.266464 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:50.266486 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:50.266918 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:50.267348 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:50.267888 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:50.269064 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:50.270200 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:50.270750 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:50.270974 (+ 224us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:50.272038 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:50.272044 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:50.272059 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:50.272665 (+ 606us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:50.272744 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:50.272784 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:50.272977 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:50.273315 (+ 338us) server_negotiation.cc:299] Negotiation successful
0619 02:33:50.273495 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":193,"threads_started":1}
I20260619 02:33:50.274688 5743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1: No bootstrap required, opened a new log
I20260619 02:33:50.278107 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:33:50.280068 5743 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7224b292e534cec9ae175ceca965be1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38185 } }
I20260619 02:33:50.280462 5743 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:50.280552 5743 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7224b292e534cec9ae175ceca965be1, State: Initialized, Role: FOLLOWER
I20260619 02:33:50.281167 5743 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7224b292e534cec9ae175ceca965be1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38185 } }
I20260619 02:33:50.281390 5743 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:33:50.281483 5743 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:33:50.281607 5743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:50.282935 5743 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7224b292e534cec9ae175ceca965be1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38185 } }
I20260619 02:33:50.283399 5743 leader_election.cc:302] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7224b292e534cec9ae175ceca965be1; no voters:
I20260619 02:33:50.283845 5743 leader_election.cc:288] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:33:50.283972 5753 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:33:50.284225 5753 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1 [term 1 LEADER]: Becoming Leader. State: Replica: c7224b292e534cec9ae175ceca965be1, State: Running, Role: LEADER
I20260619 02:33:50.284634 5753 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7224b292e534cec9ae175ceca965be1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38185 } }
I20260619 02:33:50.285610 5743 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:33:50.286296 5753 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7224b292e534cec9ae175ceca965be1. Latest consensus state: current_term: 1 leader_uuid: "c7224b292e534cec9ae175ceca965be1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7224b292e534cec9ae175ceca965be1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38185 } } }
I20260619 02:33:50.286412 5746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:50.280846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59833 (local address 127.0.0.1:38185)
0619 02:33:50.281000 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:50.281005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:50.281263 (+ 258us) server_negotiation.cc:407] Connection header received
0619 02:33:50.281438 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:50.281442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:50.281503 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:50.281599 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:50.281993 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:50.282557 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:50.283885 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:50.284300 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:50.284418 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:50.285700 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:50.285704 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:50.285707 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:50.285728 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:50.285814 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:50.285832 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:50.285941 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:50.286161 (+ 220us) server_negotiation.cc:299] Negotiation successful
0619 02:33:50.286207 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:33:50.286496 5753 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1 [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:50.287453 5754 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7224b292e534cec9ae175ceca965be1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7224b292e534cec9ae175ceca965be1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38185 } } }
I20260619 02:33:50.288586 5754 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1 [sys.catalog]: This master's current role is: LEADER
W20260619 02:33:50.292178 5768 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:33:50.292335 5768 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:33:50.292420 5767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:33:50.293259 5767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:33:50.300319 5767 catalog_manager.cc:1347] Generated new cluster ID: 0959d5bba675415685ff07fd3ff3c24d
I20260619 02:33:50.300418 5767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:33:50.316334 5767 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:33:50.317521 5767 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:33:50.336372 5767 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7224b292e534cec9ae175ceca965be1: Generated new TSK 0
I20260619 02:33:50.337495 5767 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:33:50.345736 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 5675
W20260619 02:33:50.351198 5660 connection.cc:535] client connection to 127.0.0.1:38185 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38185): Connection reset by peer (error 104)
2026-06-19T02:33:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1592 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:33:50 dist-test-slave-4qw6 krb5kdc[5774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:33:50 dist-test-slave-4qw6 krb5kdc[5774](info): set up 2 sockets
Jun 19 02:33:50 dist-test-slave-4qw6 krb5kdc[5774](info): commencing operation
krb5kdc: starting...
W20260619 02:33:51.478767 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:33:51 dist-test-slave-4qw6 krb5kdc[5774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:33:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:33:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:33:51.623383 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40319
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:33:51.749553 5787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:51.750022 5787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:51.750124 5787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:51.754674 5787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:33:51.754807 5787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:51.754850 5787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:51.754896 5787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:33:51.754930 5787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:33:51.760993 5787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40319
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:51.762400 5787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:51.763638 5787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:51.774016 5795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:51.774019 5792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:51.774302 5787 server_base.cc:1034] running on GCE node
W20260619 02:33:51.774444 5793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:51.775194 5787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:51.776508 5787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:51.777731 5787 hybrid_clock.cc:648] HybridClock initialized: now 1781836431777713 us; error 40 us; skew 500 ppm
Jun 19 02:33:51 dist-test-slave-4qw6 krb5kdc[5774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836431, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:51.781102 5787 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:33:51.782507 5787 webserver.cc:460] Webserver started at http://127.0.0.1:34361/ using document root <none> and password file <none>
I20260619 02:33:51.783226 5787 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:51.783341 5787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:51.783613 5787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:51.786293 5787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "b6fb55357c484627878a7b002a675d0c"
format_stamp: "Formatted at 2026-06-19 02:33:51 on dist-test-slave-4qw6"
server_key: "ca81d88b00a98c92c722fae9dff2b0dc"
server_key_iv: "4a58fbfedd202d2fb24a8039075bd2b2"
server_key_version: "encryptionkey@0"
I20260619 02:33:51.787006 5787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "b6fb55357c484627878a7b002a675d0c"
format_stamp: "Formatted at 2026-06-19 02:33:51 on dist-test-slave-4qw6"
server_key: "ca81d88b00a98c92c722fae9dff2b0dc"
server_key_iv: "4a58fbfedd202d2fb24a8039075bd2b2"
server_key_version: "encryptionkey@0"
I20260619 02:33:51.792079 5787 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:33:51.795266 5802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:51.797013 5787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:33:51.797206 5787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "b6fb55357c484627878a7b002a675d0c"
format_stamp: "Formatted at 2026-06-19 02:33:51 on dist-test-slave-4qw6"
server_key: "ca81d88b00a98c92c722fae9dff2b0dc"
server_key_iv: "4a58fbfedd202d2fb24a8039075bd2b2"
server_key_version: "encryptionkey@0"
I20260619 02:33:51.797375 5787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:51.803020 5787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:51.806569 5787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:51.806829 5787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:51.815348 5787 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40319
I20260619 02:33:51.815460 5854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40319 every 8 connection(s)
I20260619 02:33:51.816563 5787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:33:51.819967 5855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:51.822340 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 5787
I20260619 02:33:51.822535 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:33:51.822873 17795 external_mini_cluster.cc:1428] Setting key e0abf2a12a83a6b8ed08d0c3f5d89af6
I20260619 02:33:51.827996 5855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6fb55357c484627878a7b002a675d0c: Bootstrap starting.
I20260619 02:33:51.829195 5858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:51.824528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54375 (local address 127.0.0.1:40319)
0619 02:33:51.825077 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:51.825094 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:51.825136 (+ 42us) server_negotiation.cc:407] Connection header received
0619 02:33:51.826003 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:51.826028 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:51.826390 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:51.826811 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:51.827765 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:51.827771 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:51.827784 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:51.827796 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:51.827860 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:51.827881 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:51.828066 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:51.828144 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 02:33:51.828338 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":167,"threads_started":1}
I20260619 02:33:51.830886 5855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6fb55357c484627878a7b002a675d0c: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:51.831924 5855 log.cc:824] T 00000000000000000000000000000000 P b6fb55357c484627878a7b002a675d0c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:51.832775 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:33:51.834724 5855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6fb55357c484627878a7b002a675d0c: No bootstrap required, opened a new log
I20260619 02:33:51.837146 5858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:51.835259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55033 (local address 127.0.0.1:40319)
0619 02:33:51.835416 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:51.835435 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:51.835647 (+ 212us) server_negotiation.cc:407] Connection header received
0619 02:33:51.835752 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:51.835755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:51.835797 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:51.835918 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:51.836711 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:51.836715 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:51.836719 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:51.836735 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:51.836769 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:51.836784 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:51.836899 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:51.836962 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 02:33:51.837007 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 02:33:51.839998 5855 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6fb55357c484627878a7b002a675d0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6fb55357c484627878a7b002a675d0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40319 } }
I20260619 02:33:51.840307 5855 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6fb55357c484627878a7b002a675d0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:51.840345 5855 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6fb55357c484627878a7b002a675d0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6fb55357c484627878a7b002a675d0c, State: Initialized, Role: FOLLOWER
I20260619 02:33:51.840869 5855 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6fb55357c484627878a7b002a675d0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6fb55357c484627878a7b002a675d0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40319 } }
I20260619 02:33:51.841025 5855 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6fb55357c484627878a7b002a675d0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:33:51.841133 5855 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6fb55357c484627878a7b002a675d0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:33:51.841255 5855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6fb55357c484627878a7b002a675d0c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:51.842506 5855 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6fb55357c484627878a7b002a675d0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6fb55357c484627878a7b002a675d0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40319 } }
I20260619 02:33:51.842896 5855 leader_election.cc:302] T 00000000000000000000000000000000 P b6fb55357c484627878a7b002a675d0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6fb55357c484627878a7b002a675d0c; no voters:
I20260619 02:33:51.843215 5855 leader_election.cc:288] T 00000000000000000000000000000000 P b6fb55357c484627878a7b002a675d0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:33:51.843398 5865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6fb55357c484627878a7b002a675d0c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:33:51.843667 5865 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6fb55357c484627878a7b002a675d0c [term 1 LEADER]: Becoming Leader. State: Replica: b6fb55357c484627878a7b002a675d0c, State: Running, Role: LEADER
I20260619 02:33:51.844063 5865 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6fb55357c484627878a7b002a675d0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6fb55357c484627878a7b002a675d0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40319 } }
I20260619 02:33:51.845283 5855 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6fb55357c484627878a7b002a675d0c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:33:51.845969 5866 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6fb55357c484627878a7b002a675d0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6fb55357c484627878a7b002a675d0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6fb55357c484627878a7b002a675d0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40319 } } }
I20260619 02:33:51.846017 5867 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6fb55357c484627878a7b002a675d0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6fb55357c484627878a7b002a675d0c. Latest consensus state: current_term: 1 leader_uuid: "b6fb55357c484627878a7b002a675d0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6fb55357c484627878a7b002a675d0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40319 } } }
I20260619 02:33:51.846182 5866 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6fb55357c484627878a7b002a675d0c [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:51.846302 5867 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6fb55357c484627878a7b002a675d0c [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:51.848776 5870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:33:51.853108 5870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:33:51.859561 5870 catalog_manager.cc:1347] Generated new cluster ID: f779a96b5f2243d094677b9531384201
I20260619 02:33:51.859678 5870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:33:51.867306 5870 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:33:51.867678 5870 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:33:51.876968 5870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6fb55357c484627878a7b002a675d0c: Generated new TSK 0
I20260619 02:33:51.877898 5870 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:33:51.884533 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 5787
2026-06-19T02:33:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1539 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:33:51 dist-test-slave-4qw6 krb5kdc[5886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:33:51 dist-test-slave-4qw6 krb5kdc[5886](info): set up 2 sockets
Jun 19 02:33:51 dist-test-slave-4qw6 krb5kdc[5886](info): commencing operation
krb5kdc: starting...
W20260619 02:33:53.014650 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:33:53 dist-test-slave-4qw6 krb5kdc[5886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:33:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:33:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:33:53.161059 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46795
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:33:53.289924 5899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:53.290280 5899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:53.290333 5899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:53.295045 5899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:33:53.295135 5899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:53.295157 5899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:53.295213 5899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:33:53.295253 5899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:33:53.301393 5899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:53.302974 5899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:53.304229 5899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:53.314953 5905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:53.314872 5904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:53.314872 5907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:53.315344 5899 server_base.cc:1034] running on GCE node
I20260619 02:33:53.316038 5899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:53.317354 5899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:53.318670 5899 hybrid_clock.cc:648] HybridClock initialized: now 1781836433318660 us; error 81 us; skew 500 ppm
Jun 19 02:33:53 dist-test-slave-4qw6 krb5kdc[5886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836433, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:53.322235 5899 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:33:53.323396 5899 webserver.cc:460] Webserver started at http://127.0.0.1:33675/ using document root <none> and password file <none>
I20260619 02:33:53.324080 5899 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:53.324194 5899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:53.324465 5899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:53.326968 5899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "c336b723aeb241249da5d9ab1cf89183"
format_stamp: "Formatted at 2026-06-19 02:33:53 on dist-test-slave-4qw6"
server_key: "c07c58df379321cffe3a6f8fab597a29"
server_key_iv: "6c6617ff8e4a937977055640f7903a7b"
server_key_version: "encryptionkey@0"
I20260619 02:33:53.327643 5899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "c336b723aeb241249da5d9ab1cf89183"
format_stamp: "Formatted at 2026-06-19 02:33:53 on dist-test-slave-4qw6"
server_key: "c07c58df379321cffe3a6f8fab597a29"
server_key_iv: "6c6617ff8e4a937977055640f7903a7b"
server_key_version: "encryptionkey@0"
I20260619 02:33:53.332568 5899 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:33:53.335569 5914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:53.337019 5899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:33:53.337201 5899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "c336b723aeb241249da5d9ab1cf89183"
format_stamp: "Formatted at 2026-06-19 02:33:53 on dist-test-slave-4qw6"
server_key: "c07c58df379321cffe3a6f8fab597a29"
server_key_iv: "6c6617ff8e4a937977055640f7903a7b"
server_key_version: "encryptionkey@0"
I20260619 02:33:53.337353 5899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:53.362630 5899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:53.366973 5899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:53.367251 5899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:53.375438 5899 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46795
I20260619 02:33:53.375473 5966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46795 every 8 connection(s)
I20260619 02:33:53.376590 5899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:33:53.379930 5967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:53.380133 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 5899
I20260619 02:33:53.380326 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:33:53.380678 17795 external_mini_cluster.cc:1428] Setting key ea5672f51db90be5d41045a581735003
I20260619 02:33:53.387691 5967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c336b723aeb241249da5d9ab1cf89183: Bootstrap starting.
I20260619 02:33:53.390607 5967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c336b723aeb241249da5d9ab1cf89183: Neither blocks nor log segments found. Creating new log.
Jun 19 02:33:53 dist-test-slave-4qw6 krb5kdc[5886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 02:33:53.391693 5967 log.cc:824] T 00000000000000000000000000000000 P c336b723aeb241249da5d9ab1cf89183: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:53.394363 5967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c336b723aeb241249da5d9ab1cf89183: No bootstrap required, opened a new log
I20260619 02:33:53.397233 5970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:53.382460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55009 (local address 127.0.0.1:46795)
0619 02:33:53.383015 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:53.383032 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:53.383072 (+ 40us) server_negotiation.cc:407] Connection header received
0619 02:33:53.383891 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:53.383925 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:53.384337 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:53.384736 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:53.385669 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:53.386836 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:53.387843 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:53.388468 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:53.388673 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:53.391933 (+ 3260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:53.391940 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:53.391957 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:33:53.391992 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:53.393992 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:53.394610 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:53.394616 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:53.394621 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:53.394764 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:53.395092 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:53.395095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:53.395096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:53.395546 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:53.395703 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:53.396047 (+ 344us) server_negotiation.cc:299] Negotiation successful
0619 02:33:53.396401 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":181,"threads_started":1}
I20260619 02:33:53.399194 5967 raft_consensus.cc:348] T 00000000000000000000000000000000 P c336b723aeb241249da5d9ab1cf89183 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c336b723aeb241249da5d9ab1cf89183" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46795 } }
I20260619 02:33:53.399581 5967 raft_consensus.cc:374] T 00000000000000000000000000000000 P c336b723aeb241249da5d9ab1cf89183 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:53.399695 5967 raft_consensus.cc:729] T 00000000000000000000000000000000 P c336b723aeb241249da5d9ab1cf89183 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c336b723aeb241249da5d9ab1cf89183, State: Initialized, Role: FOLLOWER
I20260619 02:33:53.400300 5967 consensus_queue.cc:260] T 00000000000000000000000000000000 P c336b723aeb241249da5d9ab1cf89183 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c336b723aeb241249da5d9ab1cf89183" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46795 } }
I20260619 02:33:53.400516 5967 raft_consensus.cc:388] T 00000000000000000000000000000000 P c336b723aeb241249da5d9ab1cf89183 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:33:53.400592 5967 raft_consensus.cc:482] T 00000000000000000000000000000000 P c336b723aeb241249da5d9ab1cf89183 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:33:53.400708 5967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c336b723aeb241249da5d9ab1cf89183 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:53.400908 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:33:53.402034 5967 raft_consensus.cc:504] T 00000000000000000000000000000000 P c336b723aeb241249da5d9ab1cf89183 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c336b723aeb241249da5d9ab1cf89183" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46795 } }
I20260619 02:33:53.402484 5967 leader_election.cc:302] T 00000000000000000000000000000000 P c336b723aeb241249da5d9ab1cf89183 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c336b723aeb241249da5d9ab1cf89183; no voters:
I20260619 02:33:53.402874 5967 leader_election.cc:288] T 00000000000000000000000000000000 P c336b723aeb241249da5d9ab1cf89183 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:33:53.403018 5976 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c336b723aeb241249da5d9ab1cf89183 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:33:53.403268 5976 raft_consensus.cc:686] T 00000000000000000000000000000000 P c336b723aeb241249da5d9ab1cf89183 [term 1 LEADER]: Becoming Leader. State: Replica: c336b723aeb241249da5d9ab1cf89183, State: Running, Role: LEADER
I20260619 02:33:53.403782 5976 consensus_queue.cc:237] T 00000000000000000000000000000000 P c336b723aeb241249da5d9ab1cf89183 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c336b723aeb241249da5d9ab1cf89183" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46795 } }
I20260619 02:33:53.404482 5967 sys_catalog.cc:564] T 00000000000000000000000000000000 P c336b723aeb241249da5d9ab1cf89183 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:33:53.406230 5977 sys_catalog.cc:455] T 00000000000000000000000000000000 P c336b723aeb241249da5d9ab1cf89183 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c336b723aeb241249da5d9ab1cf89183" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c336b723aeb241249da5d9ab1cf89183" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46795 } } }
I20260619 02:33:53.406461 5977 sys_catalog.cc:458] T 00000000000000000000000000000000 P c336b723aeb241249da5d9ab1cf89183 [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:53.407011 5990 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:33:53.407529 5978 sys_catalog.cc:455] T 00000000000000000000000000000000 P c336b723aeb241249da5d9ab1cf89183 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c336b723aeb241249da5d9ab1cf89183. Latest consensus state: current_term: 1 leader_uuid: "c336b723aeb241249da5d9ab1cf89183" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c336b723aeb241249da5d9ab1cf89183" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46795 } } }
I20260619 02:33:53.407681 5978 sys_catalog.cc:458] T 00000000000000000000000000000000 P c336b723aeb241249da5d9ab1cf89183 [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:53.411139 5990 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:33:53.414788 5970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:53.403909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44013 (local address 127.0.0.1:46795)
0619 02:33:53.404211 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:53.404216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:53.404233 (+ 17us) server_negotiation.cc:407] Connection header received
0619 02:33:53.404413 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:53.404417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:53.404480 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:53.404596 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:53.405571 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:53.406582 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:53.408022 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:53.408425 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:53.408599 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:53.410580 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:53.410585 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:53.410588 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:33:53.410625 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:53.412595 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:53.413188 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:53.413195 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:53.413197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:53.413308 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:53.413667 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:53.413671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:53.413673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:53.414000 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:53.414151 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:53.414450 (+ 299us) server_negotiation.cc:299] Negotiation successful
0619 02:33:53.414596 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191}
I20260619 02:33:53.417955 5990 catalog_manager.cc:1347] Generated new cluster ID: 804fa56fcd864839b6045e44e3d81973
I20260619 02:33:53.418052 5990 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:33:53.433008 5990 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:33:53.434681 5990 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:33:53.441948 5990 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c336b723aeb241249da5d9ab1cf89183: Generated new TSK 0
I20260619 02:33:53.443221 5990 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:33:53.455794 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 5899
2026-06-19T02:33:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1571 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:33:53 dist-test-slave-4qw6 krb5kdc[5998](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:33:53 dist-test-slave-4qw6 krb5kdc[5998](info): set up 2 sockets
Jun 19 02:33:53 dist-test-slave-4qw6 krb5kdc[5998](info): commencing operation
krb5kdc: starting...
W20260619 02:33:54.570036 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:33:54 dist-test-slave-4qw6 krb5kdc[5998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:33:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:33:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:33:54.696563 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37401
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:33:54.811702 6011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:54.812050 6011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:54.812144 6011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:54.816427 6011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:33:54.816547 6011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:54.816586 6011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:54.816653 6011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:33:54.816708 6011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:33:54.822491 6011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:54.823712 6011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:54.824854 6011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:54.833673 6019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:54.833673 6017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:54.833899 6011 server_base.cc:1034] running on GCE node
W20260619 02:33:54.833678 6016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:54.834595 6011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:54.835772 6011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:54.836941 6011 hybrid_clock.cc:648] HybridClock initialized: now 1781836434836915 us; error 44 us; skew 500 ppm
Jun 19 02:33:54 dist-test-slave-4qw6 krb5kdc[5998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836434, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:54.844883 6011 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:33:54.846033 6011 webserver.cc:460] Webserver started at http://127.0.0.1:38035/ using document root <none> and password file <none>
I20260619 02:33:54.846647 6011 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:54.846742 6011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:54.846974 6011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:54.849133 6011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "737ffbec7a984ff3ad385d77df5c9092"
format_stamp: "Formatted at 2026-06-19 02:33:54 on dist-test-slave-4qw6"
server_key: "21e6f0f4b890a06ec9dd444314381bda"
server_key_iv: "678a0c1ac1c25814d31bbbc9266b5491"
server_key_version: "encryptionkey@0"
I20260619 02:33:54.849738 6011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "737ffbec7a984ff3ad385d77df5c9092"
format_stamp: "Formatted at 2026-06-19 02:33:54 on dist-test-slave-4qw6"
server_key: "21e6f0f4b890a06ec9dd444314381bda"
server_key_iv: "678a0c1ac1c25814d31bbbc9266b5491"
server_key_version: "encryptionkey@0"
I20260619 02:33:54.853817 6011 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 02:33:54.856323 6026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:54.857548 6011 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 02:33:54.857702 6011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "737ffbec7a984ff3ad385d77df5c9092"
format_stamp: "Formatted at 2026-06-19 02:33:54 on dist-test-slave-4qw6"
server_key: "21e6f0f4b890a06ec9dd444314381bda"
server_key_iv: "678a0c1ac1c25814d31bbbc9266b5491"
server_key_version: "encryptionkey@0"
I20260619 02:33:54.857882 6011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:54.871825 6011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:54.875085 6011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:54.875316 6011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:54.883038 6011 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37401
I20260619 02:33:54.883073 6078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37401 every 8 connection(s)
I20260619 02:33:54.884644 6011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:33:54.889925 6079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:54.894182 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 6011
I20260619 02:33:54.894348 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:33:54.894639 17795 external_mini_cluster.cc:1428] Setting key 0bccdade92ba8a44e3f76e693e1231f0
I20260619 02:33:54.898902 6079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092: Bootstrap starting.
I20260619 02:33:54.902361 6079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:54.903435 6079 log.cc:824] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:54.906149 6079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092: No bootstrap required, opened a new log
I20260619 02:33:54.907315 6082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:54.896190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36777 (local address 127.0.0.1:37401)
0619 02:33:54.896613 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:54.896640 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:54.896686 (+ 46us) server_negotiation.cc:407] Connection header received
0619 02:33:54.897755 (+ 1069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:54.897810 (+ 55us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:54.898374 (+ 564us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:54.898781 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:54.899274 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:54.901099 (+ 1825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:54.902485 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:54.903303 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:54.903497 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:54.904750 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:54.904757 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:54.904778 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:54.905368 (+ 590us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:54.905448 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:54.905481 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:54.905589 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:54.905858 (+ 269us) server_negotiation.cc:299] Negotiation successful
0619 02:33:54.906145 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":138,"threads_started":1}
I20260619 02:33:54.910996 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:33:54.913518 6079 raft_consensus.cc:348] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "737ffbec7a984ff3ad385d77df5c9092" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37401 } }
I20260619 02:33:54.913965 6079 raft_consensus.cc:374] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:54.914060 6079 raft_consensus.cc:729] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 737ffbec7a984ff3ad385d77df5c9092, State: Initialized, Role: FOLLOWER
I20260619 02:33:54.914760 6079 consensus_queue.cc:260] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "737ffbec7a984ff3ad385d77df5c9092" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37401 } }
I20260619 02:33:54.914979 6079 raft_consensus.cc:388] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:33:54.915066 6079 raft_consensus.cc:482] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:33:54.915192 6079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:54.916755 6079 raft_consensus.cc:504] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "737ffbec7a984ff3ad385d77df5c9092" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37401 } }
I20260619 02:33:54.917182 6079 leader_election.cc:302] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 737ffbec7a984ff3ad385d77df5c9092; no voters:
I20260619 02:33:54.917770 6079 leader_election.cc:288] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:33:54.917920 6089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:33:54.918215 6089 raft_consensus.cc:686] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092 [term 1 LEADER]: Becoming Leader. State: Replica: 737ffbec7a984ff3ad385d77df5c9092, State: Running, Role: LEADER
I20260619 02:33:54.918635 6089 consensus_queue.cc:237] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "737ffbec7a984ff3ad385d77df5c9092" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37401 } }
I20260619 02:33:54.919420 6079 sys_catalog.cc:564] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:33:54.919902 6082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:54.913455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33201 (local address 127.0.0.1:37401)
0619 02:33:54.913593 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:54.913597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:54.913900 (+ 303us) server_negotiation.cc:407] Connection header received
0619 02:33:54.914065 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:54.914071 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:54.914125 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:54.914206 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:54.914714 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:54.915487 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:54.916663 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:54.916997 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:54.917115 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:54.919043 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:54.919047 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:54.919050 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:54.919066 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:54.919099 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:54.919113 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:54.919212 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:54.919677 (+ 465us) server_negotiation.cc:299] Negotiation successful
0619 02:33:54.919743 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 02:33:54.922206 6090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "737ffbec7a984ff3ad385d77df5c9092" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "737ffbec7a984ff3ad385d77df5c9092" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37401 } } }
I20260619 02:33:54.922595 6090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092 [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:54.922238 6091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 737ffbec7a984ff3ad385d77df5c9092. Latest consensus state: current_term: 1 leader_uuid: "737ffbec7a984ff3ad385d77df5c9092" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "737ffbec7a984ff3ad385d77df5c9092" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37401 } } }
I20260619 02:33:54.922655 6091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092 [sys.catalog]: This master's current role is: LEADER
W20260619 02:33:54.924839 6103 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:33:54.924911 6103 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:33:54.924993 6104 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:33:54.926113 6104 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:33:54.937990 6104 catalog_manager.cc:1347] Generated new cluster ID: 2d577bf8fa9f4ad28e9346eadff161c1
I20260619 02:33:54.938102 6104 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:33:54.954591 6104 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:33:54.955755 6104 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:33:54.963821 6104 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 737ffbec7a984ff3ad385d77df5c9092: Generated new TSK 0
I20260619 02:33:54.964808 6104 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:33:54.969457 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 6011
2026-06-19T02:33:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1512 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:33:55 dist-test-slave-4qw6 krb5kdc[6110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:33:55 dist-test-slave-4qw6 krb5kdc[6110](info): set up 2 sockets
Jun 19 02:33:55 dist-test-slave-4qw6 krb5kdc[6110](info): commencing operation
krb5kdc: starting...
W20260619 02:33:56.022993 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:33:56 dist-test-slave-4qw6 krb5kdc[6110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:33:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:33:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:33:56.166239 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42181
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:33:56.295969 6123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:56.296319 6123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:56.296367 6123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:56.300868 6123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:33:56.300963 6123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:56.300985 6123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:56.301004 6123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:33:56.301023 6123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:33:56.307227 6123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:56.308451 6123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:56.309578 6123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:56.319037 6129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:56.319180 6123 server_base.cc:1034] running on GCE node
W20260619 02:33:56.319041 6128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:56.319041 6131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:56.320029 6123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:56.321274 6123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:56.322449 6123 hybrid_clock.cc:648] HybridClock initialized: now 1781836436322430 us; error 37 us; skew 500 ppm
Jun 19 02:33:56 dist-test-slave-4qw6 krb5kdc[6110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836436, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:56.326016 6123 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:33:56.327440 6123 webserver.cc:460] Webserver started at http://127.0.0.1:36723/ using document root <none> and password file <none>
I20260619 02:33:56.328269 6123 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:56.328410 6123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:56.328716 6123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:56.331677 6123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "d63eb52894794cc0b20d530303df8198"
format_stamp: "Formatted at 2026-06-19 02:33:56 on dist-test-slave-4qw6"
server_key: "5bd739c5fb168366e2c3a0fb6bed994b"
server_key_iv: "20768d16d0416800815dc320237a95ad"
server_key_version: "encryptionkey@0"
I20260619 02:33:56.332422 6123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "d63eb52894794cc0b20d530303df8198"
format_stamp: "Formatted at 2026-06-19 02:33:56 on dist-test-slave-4qw6"
server_key: "5bd739c5fb168366e2c3a0fb6bed994b"
server_key_iv: "20768d16d0416800815dc320237a95ad"
server_key_version: "encryptionkey@0"
I20260619 02:33:56.338529 6123 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 02:33:56.342036 6138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:56.343920 6123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:33:56.344125 6123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "d63eb52894794cc0b20d530303df8198"
format_stamp: "Formatted at 2026-06-19 02:33:56 on dist-test-slave-4qw6"
server_key: "5bd739c5fb168366e2c3a0fb6bed994b"
server_key_iv: "20768d16d0416800815dc320237a95ad"
server_key_version: "encryptionkey@0"
I20260619 02:33:56.344293 6123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:56.349720 6123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:56.353868 6123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:56.354120 6123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:56.363055 6123 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42181
I20260619 02:33:56.363066 6190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42181 every 8 connection(s)
I20260619 02:33:56.364184 6123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:33:56.367442 6191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:56.373587 6191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d63eb52894794cc0b20d530303df8198: Bootstrap starting.
I20260619 02:33:56.374528 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 6123
I20260619 02:33:56.374696 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:33:56.375010 17795 external_mini_cluster.cc:1428] Setting key 71fd13efd13ca94cc8e98ad141c7b361
I20260619 02:33:56.376955 6191 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d63eb52894794cc0b20d530303df8198: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:56.377899 6191 log.cc:824] T 00000000000000000000000000000000 P d63eb52894794cc0b20d530303df8198: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:56.380314 6191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d63eb52894794cc0b20d530303df8198: No bootstrap required, opened a new log
I20260619 02:33:56.380856 6194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:56.376410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56607 (local address 127.0.0.1:42181)
0619 02:33:56.376895 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:56.376911 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:56.376952 (+ 41us) server_negotiation.cc:407] Connection header received
0619 02:33:56.377818 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:56.377854 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:56.378225 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:56.378600 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:56.379508 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:56.379513 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:56.379525 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:56.379537 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:56.379595 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:56.379615 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:56.379761 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:56.379874 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 02:33:56.380044 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":147,"threads_started":1}
I20260619 02:33:56.384280 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:33:56.384961 6191 raft_consensus.cc:348] T 00000000000000000000000000000000 P d63eb52894794cc0b20d530303df8198 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63eb52894794cc0b20d530303df8198" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42181 } }
I20260619 02:33:56.385231 6191 raft_consensus.cc:374] T 00000000000000000000000000000000 P d63eb52894794cc0b20d530303df8198 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:56.385294 6191 raft_consensus.cc:729] T 00000000000000000000000000000000 P d63eb52894794cc0b20d530303df8198 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63eb52894794cc0b20d530303df8198, State: Initialized, Role: FOLLOWER
I20260619 02:33:56.385843 6191 consensus_queue.cc:260] T 00000000000000000000000000000000 P d63eb52894794cc0b20d530303df8198 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63eb52894794cc0b20d530303df8198" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42181 } }
I20260619 02:33:56.386018 6191 raft_consensus.cc:388] T 00000000000000000000000000000000 P d63eb52894794cc0b20d530303df8198 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:33:56.386087 6191 raft_consensus.cc:482] T 00000000000000000000000000000000 P d63eb52894794cc0b20d530303df8198 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:33:56.386199 6191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d63eb52894794cc0b20d530303df8198 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:56.387614 6191 raft_consensus.cc:504] T 00000000000000000000000000000000 P d63eb52894794cc0b20d530303df8198 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63eb52894794cc0b20d530303df8198" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42181 } }
I20260619 02:33:56.388022 6191 leader_election.cc:302] T 00000000000000000000000000000000 P d63eb52894794cc0b20d530303df8198 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d63eb52894794cc0b20d530303df8198; no voters:
I20260619 02:33:56.388427 6191 leader_election.cc:288] T 00000000000000000000000000000000 P d63eb52894794cc0b20d530303df8198 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:33:56.388496 6201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d63eb52894794cc0b20d530303df8198 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:33:56.388670 6194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:56.386773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35111 (local address 127.0.0.1:42181)
0619 02:33:56.386924 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:56.386929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:56.387064 (+ 135us) server_negotiation.cc:407] Connection header received
0619 02:33:56.387193 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:56.387196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:56.387230 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:56.387300 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:56.388150 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:56.388154 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:56.388159 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:56.388173 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:56.388208 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:56.388222 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:56.388310 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:56.388449 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 02:33:56.388496 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 02:33:56.388763 6201 raft_consensus.cc:686] T 00000000000000000000000000000000 P d63eb52894794cc0b20d530303df8198 [term 1 LEADER]: Becoming Leader. State: Replica: d63eb52894794cc0b20d530303df8198, State: Running, Role: LEADER
I20260619 02:33:56.389109 6201 consensus_queue.cc:237] T 00000000000000000000000000000000 P d63eb52894794cc0b20d530303df8198 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63eb52894794cc0b20d530303df8198" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42181 } }
I20260619 02:33:56.389971 6191 sys_catalog.cc:564] T 00000000000000000000000000000000 P d63eb52894794cc0b20d530303df8198 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:33:56.391284 6203 sys_catalog.cc:455] T 00000000000000000000000000000000 P d63eb52894794cc0b20d530303df8198 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d63eb52894794cc0b20d530303df8198. Latest consensus state: current_term: 1 leader_uuid: "d63eb52894794cc0b20d530303df8198" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63eb52894794cc0b20d530303df8198" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42181 } } }
I20260619 02:33:56.391299 6202 sys_catalog.cc:455] T 00000000000000000000000000000000 P d63eb52894794cc0b20d530303df8198 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d63eb52894794cc0b20d530303df8198" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63eb52894794cc0b20d530303df8198" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42181 } } }
I20260619 02:33:56.391458 6202 sys_catalog.cc:458] T 00000000000000000000000000000000 P d63eb52894794cc0b20d530303df8198 [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:56.391456 6203 sys_catalog.cc:458] T 00000000000000000000000000000000 P d63eb52894794cc0b20d530303df8198 [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:56.391889 6210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:33:56.395527 6210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:33:56.402138 6210 catalog_manager.cc:1347] Generated new cluster ID: fb2ac378bc504c819adfe2ccc0f65936
I20260619 02:33:56.402233 6210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:33:56.413044 6210 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:33:56.413424 6210 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:33:56.418577 6210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d63eb52894794cc0b20d530303df8198: Generated new TSK 0
I20260619 02:33:56.419416 6210 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:33:56.430909 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 6123
2026-06-19T02:33:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1461 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:33:56 dist-test-slave-4qw6 krb5kdc[6222](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:33:56 dist-test-slave-4qw6 krb5kdc[6222](info): set up 2 sockets
Jun 19 02:33:56 dist-test-slave-4qw6 krb5kdc[6222](info): commencing operation
krb5kdc: starting...
W20260619 02:33:57.502624 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:33:57 dist-test-slave-4qw6 krb5kdc[6222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:33:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:33:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:33:57.641228 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41611
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:33:57.755462 6235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:57.755836 6235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:57.755914 6235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:57.760131 6235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:33:57.760265 6235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:57.760308 6235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:57.760347 6235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:33:57.760380 6235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:33:57.765965 6235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:57.767244 6235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:57.768378 6235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:57.777667 6241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:57.777896 6235 server_base.cc:1034] running on GCE node
W20260619 02:33:57.777824 6243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:57.777657 6240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:57.778721 6235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:57.779956 6235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:57.781149 6235 hybrid_clock.cc:648] HybridClock initialized: now 1781836437781123 us; error 47 us; skew 500 ppm
Jun 19 02:33:57 dist-test-slave-4qw6 krb5kdc[6222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836437, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:57.784039 6235 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:33:57.785094 6235 webserver.cc:460] Webserver started at http://127.0.0.1:40183/ using document root <none> and password file <none>
I20260619 02:33:57.785722 6235 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:57.785877 6235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:57.786129 6235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:57.788323 6235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "2e4e8637b9784433867fbf377ea21bb4"
format_stamp: "Formatted at 2026-06-19 02:33:57 on dist-test-slave-4qw6"
server_key: "74e55ba70b81a39ebff74cbccdd2865b"
server_key_iv: "b5c30d4ff73326b71ff77bab19b9e9b6"
server_key_version: "encryptionkey@0"
I20260619 02:33:57.788962 6235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "2e4e8637b9784433867fbf377ea21bb4"
format_stamp: "Formatted at 2026-06-19 02:33:57 on dist-test-slave-4qw6"
server_key: "74e55ba70b81a39ebff74cbccdd2865b"
server_key_iv: "b5c30d4ff73326b71ff77bab19b9e9b6"
server_key_version: "encryptionkey@0"
I20260619 02:33:57.793120 6235 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 02:33:57.795819 6250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:57.797199 6235 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 02:33:57.797374 6235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "2e4e8637b9784433867fbf377ea21bb4"
format_stamp: "Formatted at 2026-06-19 02:33:57 on dist-test-slave-4qw6"
server_key: "74e55ba70b81a39ebff74cbccdd2865b"
server_key_iv: "b5c30d4ff73326b71ff77bab19b9e9b6"
server_key_version: "encryptionkey@0"
I20260619 02:33:57.797505 6235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:57.809157 6235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:57.812223 6235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:57.812485 6235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:57.820323 6235 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41611
I20260619 02:33:57.820350 6302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41611 every 8 connection(s)
I20260619 02:33:57.821376 6235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:33:57.824208 6303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:57.828240 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 6235
I20260619 02:33:57.828375 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:33:57.828683 17795 external_mini_cluster.cc:1428] Setting key 5ecf718d21ab89b495dd6696e7f8ac71
I20260619 02:33:57.830355 6303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4: Bootstrap starting.
I20260619 02:33:57.832675 6303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:57.833518 6303 log.cc:824] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:57.835572 6303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4: No bootstrap required, opened a new log
Jun 19 02:33:57 dist-test-slave-4qw6 krb5kdc[6222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 02:33:57.840189 6303 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e4e8637b9784433867fbf377ea21bb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41611 } }
I20260619 02:33:57.840431 6303 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:57.840492 6303 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e4e8637b9784433867fbf377ea21bb4, State: Initialized, Role: FOLLOWER
I20260619 02:33:57.840989 6303 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e4e8637b9784433867fbf377ea21bb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41611 } }
I20260619 02:33:57.841144 6303 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:33:57.841219 6303 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:33:57.841329 6303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:57.842582 6303 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e4e8637b9784433867fbf377ea21bb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41611 } }
I20260619 02:33:57.842994 6303 leader_election.cc:302] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e4e8637b9784433867fbf377ea21bb4; no voters:
I20260619 02:33:57.843384 6303 leader_election.cc:288] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:33:57.843478 6308 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:33:57.843744 6308 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4 [term 1 LEADER]: Becoming Leader. State: Replica: 2e4e8637b9784433867fbf377ea21bb4, State: Running, Role: LEADER
I20260619 02:33:57.844134 6308 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e4e8637b9784433867fbf377ea21bb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41611 } }
I20260619 02:33:57.844769 6303 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:33:57.845089 6306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:57.830230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.36:40549 (local address 127.0.0.1:41611)
0619 02:33:57.830678 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:57.830695 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:57.830732 (+ 37us) server_negotiation.cc:407] Connection header received
0619 02:33:57.831406 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:57.831424 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:57.831777 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:57.832150 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:57.833025 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:57.834423 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:57.835346 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:57.835898 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:57.836073 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:57.839081 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:57.839087 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:57.839102 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:33:57.839131 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:57.841009 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:57.841631 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:57.841636 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:57.841640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:57.841806 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:57.842257 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:57.842260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:57.842261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:57.842802 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:57.842980 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:57.843871 (+ 891us) server_negotiation.cc:299] Negotiation successful
0619 02:33:57.844110 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":155,"threads_started":1}
I20260619 02:33:57.846952 6309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e4e8637b9784433867fbf377ea21bb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e4e8637b9784433867fbf377ea21bb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41611 } } }
I20260619 02:33:57.847084 6309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4 [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:57.847681 6310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e4e8637b9784433867fbf377ea21bb4. Latest consensus state: current_term: 1 leader_uuid: "2e4e8637b9784433867fbf377ea21bb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e4e8637b9784433867fbf377ea21bb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41611 } } }
I20260619 02:33:57.847842 6310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4 [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:57.848963 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260619 02:33:57.849960 6322 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:33:57.850046 6322 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:33:57.850163 6323 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:33:57.851112 6323 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:33:57.857369 6323 catalog_manager.cc:1347] Generated new cluster ID: f4ae766c5a3c4f67aa73bf0fc6a39918
I20260619 02:33:57.857546 6323 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:33:57.861099 6306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:57.851346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.36:50663 (local address 127.0.0.1:41611)
0619 02:33:57.851466 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:57.851471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:57.851783 (+ 312us) server_negotiation.cc:407] Connection header received
0619 02:33:57.851903 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:57.851907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:57.851961 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:57.852052 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:57.852962 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:57.853820 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:57.855106 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:57.855494 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:57.855622 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:57.857254 (+ 1632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:57.857260 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:57.857263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:33:57.857290 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:57.859120 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:57.859741 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:57.859744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:57.859746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:57.859833 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:33:57.860194 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:33:57.860196 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:33:57.860198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:33:57.860384 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:57.860513 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:57.860842 (+ 329us) server_negotiation.cc:299] Negotiation successful
0619 02:33:57.860939 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 02:33:57.883069 6323 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:33:57.884119 6323 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:33:57.894353 6323 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e4e8637b9784433867fbf377ea21bb4: Generated new TSK 0
I20260619 02:33:57.895315 6323 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:33:57.899183 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 6235
W20260619 02:33:57.905413 6220 connection.cc:535] client connection to 127.0.0.1:41611 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41611): Connection reset by peer (error 104)
2026-06-19T02:33:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1468 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:33:57 dist-test-slave-4qw6 krb5kdc[6334](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:33:57 dist-test-slave-4qw6 krb5kdc[6334](info): set up 2 sockets
Jun 19 02:33:57 dist-test-slave-4qw6 krb5kdc[6334](info): commencing operation
krb5kdc: starting...
W20260619 02:33:58.938277 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:33:59 dist-test-slave-4qw6 krb5kdc[6334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:33:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:33:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:33:59.068780 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35967
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:33:59.184765 6347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:33:59.185168 6347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:33:59.185266 6347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:33:59.189567 6347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:33:59.189734 6347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:33:59.189806 6347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:33:59.189877 6347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:33:59.189918 6347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:33:59.195814 6347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:33:59.197091 6347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:33:59.198231 6347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:33:59.207598 6353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:33:59.207598 6352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:59.208021 6347 server_base.cc:1034] running on GCE node
W20260619 02:33:59.208241 6355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:33:59.209354 6347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:33:59.210668 6347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:33:59.211862 6347 hybrid_clock.cc:648] HybridClock initialized: now 1781836439211832 us; error 49 us; skew 500 ppm
Jun 19 02:33:59 dist-test-slave-4qw6 krb5kdc[6334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836439, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:33:59.214978 6347 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:33:59.216006 6347 webserver.cc:460] Webserver started at http://127.0.0.1:36087/ using document root <none> and password file <none>
I20260619 02:33:59.216570 6347 fs_manager.cc:359] Metadata directory not provided
I20260619 02:33:59.216638 6347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:33:59.216831 6347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:33:59.219274 6347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "e52627e7601242ff9ee838c90575e22e"
format_stamp: "Formatted at 2026-06-19 02:33:59 on dist-test-slave-4qw6"
server_key: "b3534b6ad15b45707a838f713c3cdc73"
server_key_iv: "0bacf47972b695dc1d8e0b89c47b06f5"
server_key_version: "encryptionkey@0"
I20260619 02:33:59.219879 6347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "e52627e7601242ff9ee838c90575e22e"
format_stamp: "Formatted at 2026-06-19 02:33:59 on dist-test-slave-4qw6"
server_key: "b3534b6ad15b45707a838f713c3cdc73"
server_key_iv: "0bacf47972b695dc1d8e0b89c47b06f5"
server_key_version: "encryptionkey@0"
I20260619 02:33:59.224126 6347 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:33:59.227003 6362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:33:59.228343 6347 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:33:59.228511 6347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "e52627e7601242ff9ee838c90575e22e"
format_stamp: "Formatted at 2026-06-19 02:33:59 on dist-test-slave-4qw6"
server_key: "b3534b6ad15b45707a838f713c3cdc73"
server_key_iv: "0bacf47972b695dc1d8e0b89c47b06f5"
server_key_version: "encryptionkey@0"
I20260619 02:33:59.228631 6347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:33:59.255141 6347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:33:59.259021 6347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:33:59.259214 6347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:33:59.267524 6347 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35967
I20260619 02:33:59.267551 6414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35967 every 8 connection(s)
I20260619 02:33:59.268698 6347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:33:59.271688 6415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:33:59.276185 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 6347
I20260619 02:33:59.276348 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:33:59.276607 17795 external_mini_cluster.cc:1428] Setting key 99796140fb716f5a50a9a55b1616f659
I20260619 02:33:59.278242 6415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e52627e7601242ff9ee838c90575e22e: Bootstrap starting.
I20260619 02:33:59.280536 6415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e52627e7601242ff9ee838c90575e22e: Neither blocks nor log segments found. Creating new log.
I20260619 02:33:59.281579 6415 log.cc:824] T 00000000000000000000000000000000 P e52627e7601242ff9ee838c90575e22e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:33:59.283646 6415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e52627e7601242ff9ee838c90575e22e: No bootstrap required, opened a new log
I20260619 02:33:59.286620 6418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:59.278240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.36:51821 (local address 127.0.0.1:35967)
0619 02:33:59.278752 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:59.278767 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:59.278806 (+ 39us) server_negotiation.cc:407] Connection header received
0619 02:33:59.279459 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:59.279475 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:59.279836 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:59.280206 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:59.280655 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:59.282223 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:59.283198 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:59.283684 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:59.283887 (+ 203us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:59.284834 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:59.284840 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:59.284853 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:59.285249 (+ 396us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:59.285305 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:59.285329 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:59.285495 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:59.285688 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 02:33:59.285873 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":154,"threads_started":1}
I20260619 02:33:59.287835 6415 raft_consensus.cc:348] T 00000000000000000000000000000000 P e52627e7601242ff9ee838c90575e22e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e52627e7601242ff9ee838c90575e22e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35967 } }
I20260619 02:33:59.288108 6415 raft_consensus.cc:374] T 00000000000000000000000000000000 P e52627e7601242ff9ee838c90575e22e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:33:59.288199 6415 raft_consensus.cc:729] T 00000000000000000000000000000000 P e52627e7601242ff9ee838c90575e22e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e52627e7601242ff9ee838c90575e22e, State: Initialized, Role: FOLLOWER
I20260619 02:33:59.288704 6415 consensus_queue.cc:260] T 00000000000000000000000000000000 P e52627e7601242ff9ee838c90575e22e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e52627e7601242ff9ee838c90575e22e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35967 } }
I20260619 02:33:59.288885 6415 raft_consensus.cc:388] T 00000000000000000000000000000000 P e52627e7601242ff9ee838c90575e22e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:33:59.288995 6415 raft_consensus.cc:482] T 00000000000000000000000000000000 P e52627e7601242ff9ee838c90575e22e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:33:59.289111 6415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e52627e7601242ff9ee838c90575e22e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:33:59.290262 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:33:59.290352 6415 raft_consensus.cc:504] T 00000000000000000000000000000000 P e52627e7601242ff9ee838c90575e22e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e52627e7601242ff9ee838c90575e22e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35967 } }
I20260619 02:33:59.290719 6415 leader_election.cc:302] T 00000000000000000000000000000000 P e52627e7601242ff9ee838c90575e22e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e52627e7601242ff9ee838c90575e22e; no voters:
I20260619 02:33:59.291076 6415 leader_election.cc:288] T 00000000000000000000000000000000 P e52627e7601242ff9ee838c90575e22e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:33:59.291155 6420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e52627e7601242ff9ee838c90575e22e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:33:59.291379 6420 raft_consensus.cc:686] T 00000000000000000000000000000000 P e52627e7601242ff9ee838c90575e22e [term 1 LEADER]: Becoming Leader. State: Replica: e52627e7601242ff9ee838c90575e22e, State: Running, Role: LEADER
I20260619 02:33:59.291765 6420 consensus_queue.cc:237] T 00000000000000000000000000000000 P e52627e7601242ff9ee838c90575e22e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e52627e7601242ff9ee838c90575e22e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35967 } }
I20260619 02:33:59.292472 6415 sys_catalog.cc:564] T 00000000000000000000000000000000 P e52627e7601242ff9ee838c90575e22e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:33:59.293473 6422 sys_catalog.cc:455] T 00000000000000000000000000000000 P e52627e7601242ff9ee838c90575e22e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e52627e7601242ff9ee838c90575e22e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e52627e7601242ff9ee838c90575e22e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35967 } } }
I20260619 02:33:59.293601 6422 sys_catalog.cc:458] T 00000000000000000000000000000000 P e52627e7601242ff9ee838c90575e22e [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:59.293977 6421 sys_catalog.cc:455] T 00000000000000000000000000000000 P e52627e7601242ff9ee838c90575e22e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e52627e7601242ff9ee838c90575e22e. Latest consensus state: current_term: 1 leader_uuid: "e52627e7601242ff9ee838c90575e22e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e52627e7601242ff9ee838c90575e22e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35967 } } }
I20260619 02:33:59.294121 6421 sys_catalog.cc:458] T 00000000000000000000000000000000 P e52627e7601242ff9ee838c90575e22e [sys.catalog]: This master's current role is: LEADER
I20260619 02:33:59.295105 6434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:33:59.297658 6418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:33:59.292528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.36:53911 (local address 127.0.0.1:35967)
0619 02:33:59.292665 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 02:33:59.292670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:33:59.292967 (+ 297us) server_negotiation.cc:407] Connection header received
0619 02:33:59.293156 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:33:59.293161 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:33:59.293212 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:33:59.293264 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:33:59.293682 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:59.294469 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:59.295727 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:33:59.296048 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:33:59.296186 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:33:59.297162 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:33:59.297166 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:33:59.297169 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:33:59.297183 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:33:59.297217 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:33:59.297236 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:33:59.297345 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 02:33:59.297469 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 02:33:59.297505 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:33:59.298359 6434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:33:59.304867 6434 catalog_manager.cc:1347] Generated new cluster ID: 00f22ab8c0584949b5abd7b6e4ecfac6
I20260619 02:33:59.304983 6434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:33:59.312371 6434 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:33:59.313432 6434 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:33:59.326234 6434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e52627e7601242ff9ee838c90575e22e: Generated new TSK 0
I20260619 02:33:59.327215 6434 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:33:59.334839 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 6347
W20260619 02:33:59.342584 6332 connection.cc:535] client connection to 127.0.0.1:35967 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35967): Connection reset by peer (error 104)
2026-06-19T02:33:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1437 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:33:59 dist-test-slave-4qw6 krb5kdc[6446](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:33:59 dist-test-slave-4qw6 krb5kdc[6446](info): set up 2 sockets
Jun 19 02:33:59 dist-test-slave-4qw6 krb5kdc[6446](info): commencing operation
krb5kdc: starting...
W20260619 02:34:00.386229 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:34:00 dist-test-slave-4qw6 krb5kdc[6446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:34:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:34:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:34:00.516644 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41371
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:34:00.632997 6459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:34:00.633392 6459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:34:00.633468 6459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:34:00.637835 6459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:34:00.637982 6459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:34:00.638020 6459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:34:00.638062 6459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:34:00.638098 6459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:34:00.644011 6459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:34:00.645287 6459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:34:00.646435 6459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:34:00.655719 6464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:00.655827 6467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:00.655719 6465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:34:00.655997 6459 server_base.cc:1034] running on GCE node
I20260619 02:34:00.656669 6459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:34:00.657923 6459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:34:00.659106 6459 hybrid_clock.cc:648] HybridClock initialized: now 1781836440659092 us; error 30 us; skew 500 ppm
Jun 19 02:34:00 dist-test-slave-4qw6 krb5kdc[6446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836440, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:34:00.665977 6459 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:34:00.667112 6459 webserver.cc:460] Webserver started at http://127.0.0.1:46045/ using document root <none> and password file <none>
I20260619 02:34:00.667762 6459 fs_manager.cc:359] Metadata directory not provided
I20260619 02:34:00.667941 6459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:34:00.668202 6459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:34:00.670545 6459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "da45d64e115c476088bcb000ba3c4dbc"
format_stamp: "Formatted at 2026-06-19 02:34:00 on dist-test-slave-4qw6"
server_key: "94bd75ea6c4e5dd8409d366a4489ebe8"
server_key_iv: "e1f97faacb14f97ea822aaa18f7b2758"
server_key_version: "encryptionkey@0"
I20260619 02:34:00.671164 6459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "da45d64e115c476088bcb000ba3c4dbc"
format_stamp: "Formatted at 2026-06-19 02:34:00 on dist-test-slave-4qw6"
server_key: "94bd75ea6c4e5dd8409d366a4489ebe8"
server_key_iv: "e1f97faacb14f97ea822aaa18f7b2758"
server_key_version: "encryptionkey@0"
I20260619 02:34:00.675235 6459 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 02:34:00.678066 6474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:34:00.679468 6459 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 02:34:00.679657 6459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "da45d64e115c476088bcb000ba3c4dbc"
format_stamp: "Formatted at 2026-06-19 02:34:00 on dist-test-slave-4qw6"
server_key: "94bd75ea6c4e5dd8409d366a4489ebe8"
server_key_iv: "e1f97faacb14f97ea822aaa18f7b2758"
server_key_version: "encryptionkey@0"
I20260619 02:34:00.679811 6459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:34:00.684957 6459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:34:00.688215 6459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:34:00.688494 6459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:34:00.696420 6459 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41371
I20260619 02:34:00.696446 6526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41371 every 8 connection(s)
I20260619 02:34:00.697484 6459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:34:00.700479 6527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:34:00.703589 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 6459
I20260619 02:34:00.703756 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:34:00.704079 17795 external_mini_cluster.cc:1428] Setting key be975fc0466477f26ab71c406ea3c1c2
I20260619 02:34:00.707496 6527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da45d64e115c476088bcb000ba3c4dbc: Bootstrap starting.
I20260619 02:34:00.709899 6530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:00.705796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.36:58855 (local address 127.0.0.1:41371)
0619 02:34:00.706316 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:00.706333 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:00.706372 (+ 39us) server_negotiation.cc:407] Connection header received
0619 02:34:00.707011 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:00.707030 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:00.707344 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:00.707713 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:00.708583 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:00.708588 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:00.708600 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:34:00.708610 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:00.708665 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:34:00.708685 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:00.708847 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:00.708919 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 02:34:00.709091 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":153,"threads_started":1}
I20260619 02:34:00.710434 6527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da45d64e115c476088bcb000ba3c4dbc: Neither blocks nor log segments found. Creating new log.
I20260619 02:34:00.711338 6527 log.cc:824] T 00000000000000000000000000000000 P da45d64e115c476088bcb000ba3c4dbc: Log is configured to *not* fsync() on all Append() calls
I20260619 02:34:00.713274 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:34:00.713902 6527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da45d64e115c476088bcb000ba3c4dbc: No bootstrap required, opened a new log
I20260619 02:34:00.717376 6530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:00.715687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.36:44591 (local address 127.0.0.1:41371)
0619 02:34:00.715822 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:00.715828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:00.715978 (+ 150us) server_negotiation.cc:407] Connection header received
0619 02:34:00.716079 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:00.716082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:00.716125 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:00.716222 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:00.716966 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:00.716970 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:00.716973 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:34:00.716986 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:00.717018 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:34:00.717032 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:00.717129 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:00.717183 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 02:34:00.717229 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 02:34:00.718886 6527 raft_consensus.cc:348] T 00000000000000000000000000000000 P da45d64e115c476088bcb000ba3c4dbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da45d64e115c476088bcb000ba3c4dbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41371 } }
I20260619 02:34:00.719158 6527 raft_consensus.cc:374] T 00000000000000000000000000000000 P da45d64e115c476088bcb000ba3c4dbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:34:00.719192 6527 raft_consensus.cc:729] T 00000000000000000000000000000000 P da45d64e115c476088bcb000ba3c4dbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da45d64e115c476088bcb000ba3c4dbc, State: Initialized, Role: FOLLOWER
I20260619 02:34:00.719663 6527 consensus_queue.cc:260] T 00000000000000000000000000000000 P da45d64e115c476088bcb000ba3c4dbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da45d64e115c476088bcb000ba3c4dbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41371 } }
I20260619 02:34:00.719797 6527 raft_consensus.cc:388] T 00000000000000000000000000000000 P da45d64e115c476088bcb000ba3c4dbc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:34:00.719848 6527 raft_consensus.cc:482] T 00000000000000000000000000000000 P da45d64e115c476088bcb000ba3c4dbc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:34:00.719934 6527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da45d64e115c476088bcb000ba3c4dbc [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:34:00.720930 6527 raft_consensus.cc:504] T 00000000000000000000000000000000 P da45d64e115c476088bcb000ba3c4dbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da45d64e115c476088bcb000ba3c4dbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41371 } }
I20260619 02:34:00.721264 6527 leader_election.cc:302] T 00000000000000000000000000000000 P da45d64e115c476088bcb000ba3c4dbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da45d64e115c476088bcb000ba3c4dbc; no voters:
I20260619 02:34:00.721551 6527 leader_election.cc:288] T 00000000000000000000000000000000 P da45d64e115c476088bcb000ba3c4dbc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:34:00.721691 6537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da45d64e115c476088bcb000ba3c4dbc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:34:00.722002 6537 raft_consensus.cc:686] T 00000000000000000000000000000000 P da45d64e115c476088bcb000ba3c4dbc [term 1 LEADER]: Becoming Leader. State: Replica: da45d64e115c476088bcb000ba3c4dbc, State: Running, Role: LEADER
I20260619 02:34:00.722383 6537 consensus_queue.cc:237] T 00000000000000000000000000000000 P da45d64e115c476088bcb000ba3c4dbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da45d64e115c476088bcb000ba3c4dbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41371 } }
I20260619 02:34:00.723205 6527 sys_catalog.cc:564] T 00000000000000000000000000000000 P da45d64e115c476088bcb000ba3c4dbc [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:34:00.724282 6538 sys_catalog.cc:455] T 00000000000000000000000000000000 P da45d64e115c476088bcb000ba3c4dbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da45d64e115c476088bcb000ba3c4dbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da45d64e115c476088bcb000ba3c4dbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41371 } } }
I20260619 02:34:00.724315 6539 sys_catalog.cc:455] T 00000000000000000000000000000000 P da45d64e115c476088bcb000ba3c4dbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader da45d64e115c476088bcb000ba3c4dbc. Latest consensus state: current_term: 1 leader_uuid: "da45d64e115c476088bcb000ba3c4dbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da45d64e115c476088bcb000ba3c4dbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41371 } } }
I20260619 02:34:00.724470 6538 sys_catalog.cc:458] T 00000000000000000000000000000000 P da45d64e115c476088bcb000ba3c4dbc [sys.catalog]: This master's current role is: LEADER
I20260619 02:34:00.724470 6539 sys_catalog.cc:458] T 00000000000000000000000000000000 P da45d64e115c476088bcb000ba3c4dbc [sys.catalog]: This master's current role is: LEADER
I20260619 02:34:00.724936 6546 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:34:00.728461 6546 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:34:00.735026 6546 catalog_manager.cc:1347] Generated new cluster ID: 3375abd84a4f476d89917e6a942756dc
I20260619 02:34:00.735129 6546 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:34:00.745637 6546 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:34:00.746079 6546 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:34:00.751760 6546 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da45d64e115c476088bcb000ba3c4dbc: Generated new TSK 0
I20260619 02:34:00.752713 6546 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:34:00.765291 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 6459
2026-06-19T02:34:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:34:00 dist-test-slave-4qw6 krb5kdc[6558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:34:00 dist-test-slave-4qw6 krb5kdc[6558](info): set up 2 sockets
Jun 19 02:34:00 dist-test-slave-4qw6 krb5kdc[6558](info): commencing operation
krb5kdc: starting...
W20260619 02:34:01.866891 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:34:01 dist-test-slave-4qw6 krb5kdc[6558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:34:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:34:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:34:02.014765 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46227
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:34:02.148799 6571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:34:02.149125 6571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:34:02.149171 6571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:34:02.153347 6571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:34:02.153424 6571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:34:02.153451 6571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:34:02.153469 6571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:34:02.153486 6571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:34:02.159160 6571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46227
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:34:02.160353 6571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:34:02.161406 6571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:34:02.170881 6577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:02.170873 6576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:02.170873 6579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:34:02.171589 6571 server_base.cc:1034] running on GCE node
I20260619 02:34:02.172219 6571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:34:02.173404 6571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:34:02.174584 6571 hybrid_clock.cc:648] HybridClock initialized: now 1781836442174553 us; error 48 us; skew 500 ppm
Jun 19 02:34:02 dist-test-slave-4qw6 krb5kdc[6558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836442, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:34:02.177733 6571 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:34:02.178892 6571 webserver.cc:460] Webserver started at http://127.0.0.1:35121/ using document root <none> and password file <none>
I20260619 02:34:02.179795 6571 fs_manager.cc:359] Metadata directory not provided
I20260619 02:34:02.179904 6571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:34:02.180169 6571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:34:02.183344 6571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "7d17514a95614f8e9933b9e70d240fcf"
format_stamp: "Formatted at 2026-06-19 02:34:02 on dist-test-slave-4qw6"
server_key: "e7dfff95e156c983e84c026a7bdb1011"
server_key_iv: "5b3dd2eba54a01541f3fb6c39af89029"
server_key_version: "encryptionkey@0"
I20260619 02:34:02.184032 6571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "7d17514a95614f8e9933b9e70d240fcf"
format_stamp: "Formatted at 2026-06-19 02:34:02 on dist-test-slave-4qw6"
server_key: "e7dfff95e156c983e84c026a7bdb1011"
server_key_iv: "5b3dd2eba54a01541f3fb6c39af89029"
server_key_version: "encryptionkey@0"
I20260619 02:34:02.188524 6571 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:34:02.191466 6586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:34:02.192873 6571 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:34:02.193054 6571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "7d17514a95614f8e9933b9e70d240fcf"
format_stamp: "Formatted at 2026-06-19 02:34:02 on dist-test-slave-4qw6"
server_key: "e7dfff95e156c983e84c026a7bdb1011"
server_key_iv: "5b3dd2eba54a01541f3fb6c39af89029"
server_key_version: "encryptionkey@0"
I20260619 02:34:02.193202 6571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:34:02.206883 6571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:34:02.210309 6571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:34:02.210559 6571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:34:02.218509 6571 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46227
I20260619 02:34:02.218641 6638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46227 every 8 connection(s)
I20260619 02:34:02.219651 6571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:34:02.222712 6639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:34:02.223961 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 6571
I20260619 02:34:02.224116 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:34:02.224468 17795 external_mini_cluster.cc:1428] Setting key cdf5d5bfcb7ce3a9c266284051f13a3b
I20260619 02:34:02.229628 6639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf: Bootstrap starting.
I20260619 02:34:02.232131 6639 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf: Neither blocks nor log segments found. Creating new log.
I20260619 02:34:02.232959 6639 log.cc:824] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf: Log is configured to *not* fsync() on all Append() calls
Jun 19 02:34:02 dist-test-slave-4qw6 krb5kdc[6558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 02:34:02.235385 6639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf: No bootstrap required, opened a new log
I20260619 02:34:02.239114 6639 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d17514a95614f8e9933b9e70d240fcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46227 } }
I20260619 02:34:02.239420 6639 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:34:02.239497 6639 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d17514a95614f8e9933b9e70d240fcf, State: Initialized, Role: FOLLOWER
I20260619 02:34:02.240056 6639 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d17514a95614f8e9933b9e70d240fcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46227 } }
I20260619 02:34:02.240228 6639 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:34:02.240320 6639 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:34:02.240450 6639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:34:02.240638 6642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:02.226099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.36:40925 (local address 127.0.0.1:46227)
0619 02:34:02.226611 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:02.226628 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:02.226718 (+ 90us) server_negotiation.cc:407] Connection header received
0619 02:34:02.227486 (+ 768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:02.227504 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:02.227895 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:02.228341 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:02.229196 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:02.230342 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:02.231263 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:02.231773 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:02.231973 (+ 200us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:02.235186 (+ 3213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:02.235191 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:02.235207 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:02.235236 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:02.237143 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:02.237796 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:02.237806 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:02.237812 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:02.237952 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:02.238333 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:02.238337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:02.238340 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:02.238789 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:02.239055 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:02.239364 (+ 309us) server_negotiation.cc:299] Negotiation successful
0619 02:34:02.239629 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":169,"threads_started":1}
I20260619 02:34:02.241825 6639 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d17514a95614f8e9933b9e70d240fcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46227 } }
I20260619 02:34:02.242230 6639 leader_election.cc:302] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d17514a95614f8e9933b9e70d240fcf; no voters:
I20260619 02:34:02.242610 6639 leader_election.cc:288] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:34:02.242733 6644 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:34:02.242969 6644 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf [term 1 LEADER]: Becoming Leader. State: Replica: 7d17514a95614f8e9933b9e70d240fcf, State: Running, Role: LEADER
I20260619 02:34:02.243363 6644 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d17514a95614f8e9933b9e70d240fcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46227 } }
I20260619 02:34:02.244025 6639 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:34:02.244580 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:34:02.244731 6644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d17514a95614f8e9933b9e70d240fcf. Latest consensus state: current_term: 1 leader_uuid: "7d17514a95614f8e9933b9e70d240fcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d17514a95614f8e9933b9e70d240fcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46227 } } }
I20260619 02:34:02.244855 6644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf [sys.catalog]: This master's current role is: LEADER
I20260619 02:34:02.245182 6645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d17514a95614f8e9933b9e70d240fcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d17514a95614f8e9933b9e70d240fcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46227 } } }
I20260619 02:34:02.245350 6645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf [sys.catalog]: This master's current role is: LEADER
W20260619 02:34:02.249817 6661 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:34:02.249967 6661 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:34:02.250038 6650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:34:02.250952 6650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:34:02.258675 6650 catalog_manager.cc:1347] Generated new cluster ID: dd52ecf80b464b1a9f20230a3fe90530
I20260619 02:34:02.258733 6642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:02.247474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.36:39821 (local address 127.0.0.1:46227)
0619 02:34:02.247598 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:02.247604 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:02.247850 (+ 246us) server_negotiation.cc:407] Connection header received
0619 02:34:02.248028 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:02.248031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:02.248082 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:02.248192 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:02.249105 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:02.249973 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:02.251240 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:02.251656 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:02.251784 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:02.253957 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:02.253962 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:02.253965 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:02.254001 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:02.256083 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:02.256867 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:02.256871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:02.256873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:02.256989 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:02.257481 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:02.257485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:02.257487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:02.257795 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:02.257980 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:02.258399 (+ 419us) server_negotiation.cc:299] Negotiation successful
0619 02:34:02.258534 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 02:34:02.258751 6650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:34:02.269819 6650 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:34:02.271194 6650 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:34:02.279664 6650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d17514a95614f8e9933b9e70d240fcf: Generated new TSK 0
I20260619 02:34:02.281193 6650 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:34:02.292201 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 6571
W20260619 02:34:02.299813 6556 connection.cc:535] client connection to 127.0.0.1:46227 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46227): Connection reset by peer (error 104)
2026-06-19T02:34:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1528 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:34:02 dist-test-slave-4qw6 krb5kdc[6670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:34:02 dist-test-slave-4qw6 krb5kdc[6670](info): set up 2 sockets
Jun 19 02:34:02 dist-test-slave-4qw6 krb5kdc[6670](info): commencing operation
krb5kdc: starting...
W20260619 02:34:03.394508 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:34:03 dist-test-slave-4qw6 krb5kdc[6670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:34:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:34:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:34:03.538622 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44205
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:34:03.658690 6683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:34:03.658998 6683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:34:03.659041 6683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:34:03.663148 6683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:34:03.663229 6683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:34:03.663249 6683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:34:03.663266 6683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:34:03.663281 6683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:34:03.668932 6683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44205
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:34:03.670186 6683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:34:03.671269 6683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:34:03.680621 6683 server_base.cc:1034] running on GCE node
W20260619 02:34:03.680586 6689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:03.680576 6691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:03.680576 6688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:34:03.681463 6683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:34:03.682727 6683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:34:03.683905 6683 hybrid_clock.cc:648] HybridClock initialized: now 1781836443683880 us; error 44 us; skew 500 ppm
Jun 19 02:34:03 dist-test-slave-4qw6 krb5kdc[6670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836443, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:34:03.687063 6683 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:34:03.688038 6683 webserver.cc:460] Webserver started at http://127.0.0.1:39815/ using document root <none> and password file <none>
I20260619 02:34:03.688638 6683 fs_manager.cc:359] Metadata directory not provided
I20260619 02:34:03.688704 6683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:34:03.688884 6683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:34:03.691231 6683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "0114727d101641efa0d815e92be52c7e"
format_stamp: "Formatted at 2026-06-19 02:34:03 on dist-test-slave-4qw6"
server_key: "69ed9f07e5135b808380900eb061f0fb"
server_key_iv: "d26c9cb8d0defb7cd9c30942f033501e"
server_key_version: "encryptionkey@0"
I20260619 02:34:03.691850 6683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "0114727d101641efa0d815e92be52c7e"
format_stamp: "Formatted at 2026-06-19 02:34:03 on dist-test-slave-4qw6"
server_key: "69ed9f07e5135b808380900eb061f0fb"
server_key_iv: "d26c9cb8d0defb7cd9c30942f033501e"
server_key_version: "encryptionkey@0"
I20260619 02:34:03.696730 6683 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:34:03.700294 6698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:34:03.702132 6683 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 02:34:03.702347 6683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "0114727d101641efa0d815e92be52c7e"
format_stamp: "Formatted at 2026-06-19 02:34:03 on dist-test-slave-4qw6"
server_key: "69ed9f07e5135b808380900eb061f0fb"
server_key_iv: "d26c9cb8d0defb7cd9c30942f033501e"
server_key_version: "encryptionkey@0"
I20260619 02:34:03.702510 6683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:34:03.717571 6683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:34:03.721899 6683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:34:03.722239 6683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:34:03.731266 6683 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44205
I20260619 02:34:03.731292 6750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44205 every 8 connection(s)
I20260619 02:34:03.732551 6683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:34:03.735986 6751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:34:03.737174 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 6683
I20260619 02:34:03.737355 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:34:03.737669 17795 external_mini_cluster.cc:1428] Setting key 43c7b52dcf3971aaa9aaba249a4bdad1
I20260619 02:34:03.742903 6751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0114727d101641efa0d815e92be52c7e: Bootstrap starting.
I20260619 02:34:03.745409 6751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0114727d101641efa0d815e92be52c7e: Neither blocks nor log segments found. Creating new log.
I20260619 02:34:03.746323 6751 log.cc:824] T 00000000000000000000000000000000 P 0114727d101641efa0d815e92be52c7e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:34:03.747792 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:03.739458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.36:51647 (local address 127.0.0.1:44205)
0619 02:34:03.739997 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:03.740013 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:03.740052 (+ 39us) server_negotiation.cc:407] Connection header received
0619 02:34:03.740751 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:03.740777 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:03.741153 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:03.741545 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:03.742027 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:03.743059 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:03.744060 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:03.744596 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:03.744794 (+ 198us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:03.745872 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:03.745881 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:03.745895 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:34:03.746341 (+ 446us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:03.746439 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:34:03.746478 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:03.746641 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:03.746843 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 02:34:03.747003 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":172,"threads_started":1}
I20260619 02:34:03.748561 6751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0114727d101641efa0d815e92be52c7e: No bootstrap required, opened a new log
I20260619 02:34:03.751092 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:34:03.752785 6751 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0114727d101641efa0d815e92be52c7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0114727d101641efa0d815e92be52c7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44205 } }
I20260619 02:34:03.753090 6751 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0114727d101641efa0d815e92be52c7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:34:03.753153 6751 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0114727d101641efa0d815e92be52c7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0114727d101641efa0d815e92be52c7e, State: Initialized, Role: FOLLOWER
I20260619 02:34:03.753675 6751 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0114727d101641efa0d815e92be52c7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0114727d101641efa0d815e92be52c7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44205 } }
I20260619 02:34:03.753909 6751 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0114727d101641efa0d815e92be52c7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:34:03.753995 6751 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0114727d101641efa0d815e92be52c7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:34:03.754101 6751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0114727d101641efa0d815e92be52c7e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:34:03.755162 6751 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0114727d101641efa0d815e92be52c7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0114727d101641efa0d815e92be52c7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44205 } }
I20260619 02:34:03.755560 6751 leader_election.cc:302] T 00000000000000000000000000000000 P 0114727d101641efa0d815e92be52c7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0114727d101641efa0d815e92be52c7e; no voters:
I20260619 02:34:03.755998 6751 leader_election.cc:288] T 00000000000000000000000000000000 P 0114727d101641efa0d815e92be52c7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:34:03.756079 6761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0114727d101641efa0d815e92be52c7e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:34:03.756292 6761 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0114727d101641efa0d815e92be52c7e [term 1 LEADER]: Becoming Leader. State: Replica: 0114727d101641efa0d815e92be52c7e, State: Running, Role: LEADER
I20260619 02:34:03.756718 6761 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0114727d101641efa0d815e92be52c7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0114727d101641efa0d815e92be52c7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44205 } }
I20260619 02:34:03.757519 6751 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0114727d101641efa0d815e92be52c7e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:34:03.758069 6763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0114727d101641efa0d815e92be52c7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0114727d101641efa0d815e92be52c7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0114727d101641efa0d815e92be52c7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44205 } } }
I20260619 02:34:03.758293 6763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0114727d101641efa0d815e92be52c7e [sys.catalog]: This master's current role is: LEADER
I20260619 02:34:03.758522 6762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0114727d101641efa0d815e92be52c7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0114727d101641efa0d815e92be52c7e. Latest consensus state: current_term: 1 leader_uuid: "0114727d101641efa0d815e92be52c7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0114727d101641efa0d815e92be52c7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44205 } } }
I20260619 02:34:03.758656 6762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0114727d101641efa0d815e92be52c7e [sys.catalog]: This master's current role is: LEADER
I20260619 02:34:03.758708 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:03.753450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.36:48879 (local address 127.0.0.1:44205)
0619 02:34:03.753572 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:03.753577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:03.753868 (+ 291us) server_negotiation.cc:407] Connection header received
0619 02:34:03.754047 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:03.754050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:03.754106 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:03.754187 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:03.754516 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:03.755106 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:03.756093 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:03.756489 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:03.756615 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:03.757679 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:03.757682 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:03.757685 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:34:03.757701 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:03.757739 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:34:03.757811 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:03.757915 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:03.758444 (+ 529us) server_negotiation.cc:299] Negotiation successful
0619 02:34:03.758532 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 02:34:03.758831 6766 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:34:03.763754 6766 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:34:03.770673 6766 catalog_manager.cc:1347] Generated new cluster ID: 46572d657b084fe8a10a8fb6d87abc80
I20260619 02:34:03.770777 6766 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:34:03.789438 6766 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:34:03.790623 6766 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:34:03.798556 6766 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0114727d101641efa0d815e92be52c7e: Generated new TSK 0
I20260619 02:34:03.799535 6766 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:34:03.810779 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 6683
W20260619 02:34:03.816653 6668 connection.cc:535] client connection to 127.0.0.1:44205 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44205): Connection reset by peer (error 104)
2026-06-19T02:34:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1516 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:34:03 dist-test-slave-4qw6 krb5kdc[6782](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:34:03 dist-test-slave-4qw6 krb5kdc[6782](info): set up 2 sockets
Jun 19 02:34:03 dist-test-slave-4qw6 krb5kdc[6782](info): commencing operation
krb5kdc: starting...
W20260619 02:34:04.906247 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:34:05 dist-test-slave-4qw6 krb5kdc[6782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:34:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:34:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:34:05.061697 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33031
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:34:05.187158 6795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:34:05.187505 6795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:34:05.187552 6795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:34:05.192191 6795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:34:05.192286 6795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:34:05.192306 6795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:34:05.192323 6795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:34:05.192339 6795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:34:05.199509 6795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33031
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:34:05.200666 6795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:34:05.201668 6795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:34:05.210928 6801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:05.210930 6803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:05.210952 6800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:34:05.211278 6795 server_base.cc:1034] running on GCE node
I20260619 02:34:05.211964 6795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:34:05.213220 6795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:34:05.214401 6795 hybrid_clock.cc:648] HybridClock initialized: now 1781836445214382 us; error 35 us; skew 500 ppm
Jun 19 02:34:05 dist-test-slave-4qw6 krb5kdc[6782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836445, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:34:05.217522 6795 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:34:05.218612 6795 webserver.cc:460] Webserver started at http://127.0.0.1:46557/ using document root <none> and password file <none>
I20260619 02:34:05.219265 6795 fs_manager.cc:359] Metadata directory not provided
I20260619 02:34:05.219369 6795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:34:05.219604 6795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:34:05.221870 6795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "5fca8b4b81d546ac99f0c7de11abf658"
format_stamp: "Formatted at 2026-06-19 02:34:05 on dist-test-slave-4qw6"
server_key: "50ef377707216aee4e4477cddd135087"
server_key_iv: "5c1645b28fd2e8b6d568f7aca64f9087"
server_key_version: "encryptionkey@0"
I20260619 02:34:05.222496 6795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "5fca8b4b81d546ac99f0c7de11abf658"
format_stamp: "Formatted at 2026-06-19 02:34:05 on dist-test-slave-4qw6"
server_key: "50ef377707216aee4e4477cddd135087"
server_key_iv: "5c1645b28fd2e8b6d568f7aca64f9087"
server_key_version: "encryptionkey@0"
I20260619 02:34:05.226616 6795 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260619 02:34:05.229357 6810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:34:05.230896 6795 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 02:34:05.231047 6795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "5fca8b4b81d546ac99f0c7de11abf658"
format_stamp: "Formatted at 2026-06-19 02:34:05 on dist-test-slave-4qw6"
server_key: "50ef377707216aee4e4477cddd135087"
server_key_iv: "5c1645b28fd2e8b6d568f7aca64f9087"
server_key_version: "encryptionkey@0"
I20260619 02:34:05.231180 6795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:34:05.236439 6795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:34:05.240301 6795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:34:05.240494 6795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:34:05.249289 6795 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33031
I20260619 02:34:05.249311 6862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33031 every 8 connection(s)
I20260619 02:34:05.250551 6795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:34:05.253854 6863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:34:05.260321 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 6795
I20260619 02:34:05.260458 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:34:05.260653 6863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fca8b4b81d546ac99f0c7de11abf658: Bootstrap starting.
I20260619 02:34:05.260762 17795 external_mini_cluster.cc:1428] Setting key 7ac51d5d2d0b40c4646e5de7f7397aad
I20260619 02:34:05.263470 6863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fca8b4b81d546ac99f0c7de11abf658: Neither blocks nor log segments found. Creating new log.
I20260619 02:34:05.264310 6863 log.cc:824] T 00000000000000000000000000000000 P 5fca8b4b81d546ac99f0c7de11abf658: Log is configured to *not* fsync() on all Append() calls
I20260619 02:34:05.266476 6866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:05.262277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.36:46935 (local address 127.0.0.1:33031)
0619 02:34:05.262779 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:05.262795 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:05.262833 (+ 38us) server_negotiation.cc:407] Connection header received
0619 02:34:05.263529 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:05.263547 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:05.263946 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:05.264320 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:05.265171 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:05.265175 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:05.265187 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:34:05.265198 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:05.265252 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:34:05.265271 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:05.265412 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:05.265488 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 02:34:05.265662 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":150,"threads_started":1}
I20260619 02:34:05.266722 6863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fca8b4b81d546ac99f0c7de11abf658: No bootstrap required, opened a new log
I20260619 02:34:05.269876 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:34:05.271384 6863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fca8b4b81d546ac99f0c7de11abf658 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fca8b4b81d546ac99f0c7de11abf658" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33031 } }
I20260619 02:34:05.271684 6863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5fca8b4b81d546ac99f0c7de11abf658 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:34:05.271780 6863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fca8b4b81d546ac99f0c7de11abf658 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fca8b4b81d546ac99f0c7de11abf658, State: Initialized, Role: FOLLOWER
I20260619 02:34:05.272351 6863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fca8b4b81d546ac99f0c7de11abf658 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fca8b4b81d546ac99f0c7de11abf658" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33031 } }
I20260619 02:34:05.272547 6863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5fca8b4b81d546ac99f0c7de11abf658 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:34:05.272620 6863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fca8b4b81d546ac99f0c7de11abf658 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:34:05.272727 6863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fca8b4b81d546ac99f0c7de11abf658 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:34:05.274031 6863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fca8b4b81d546ac99f0c7de11abf658 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fca8b4b81d546ac99f0c7de11abf658" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33031 } }
I20260619 02:34:05.274225 6866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:05.272166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.36:52565 (local address 127.0.0.1:33031)
0619 02:34:05.272317 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:05.272323 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:05.272481 (+ 158us) server_negotiation.cc:407] Connection header received
0619 02:34:05.272659 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:05.272664 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:05.272740 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:05.272821 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:05.273702 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:05.273705 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:05.273708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:34:05.273720 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:05.273775 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:34:05.273789 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:05.273880 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:05.273972 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 02:34:05.274021 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 02:34:05.274530 6863 leader_election.cc:302] T 00000000000000000000000000000000 P 5fca8b4b81d546ac99f0c7de11abf658 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5fca8b4b81d546ac99f0c7de11abf658; no voters:
I20260619 02:34:05.274981 6863 leader_election.cc:288] T 00000000000000000000000000000000 P 5fca8b4b81d546ac99f0c7de11abf658 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:34:05.275098 6873 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fca8b4b81d546ac99f0c7de11abf658 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:34:05.275362 6873 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fca8b4b81d546ac99f0c7de11abf658 [term 1 LEADER]: Becoming Leader. State: Replica: 5fca8b4b81d546ac99f0c7de11abf658, State: Running, Role: LEADER
I20260619 02:34:05.275720 6873 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fca8b4b81d546ac99f0c7de11abf658 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fca8b4b81d546ac99f0c7de11abf658" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33031 } }
I20260619 02:34:05.276417 6863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fca8b4b81d546ac99f0c7de11abf658 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:34:05.277735 6874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fca8b4b81d546ac99f0c7de11abf658 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5fca8b4b81d546ac99f0c7de11abf658" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fca8b4b81d546ac99f0c7de11abf658" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33031 } } }
I20260619 02:34:05.277948 6874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fca8b4b81d546ac99f0c7de11abf658 [sys.catalog]: This master's current role is: LEADER
I20260619 02:34:05.277793 6875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fca8b4b81d546ac99f0c7de11abf658 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fca8b4b81d546ac99f0c7de11abf658. Latest consensus state: current_term: 1 leader_uuid: "5fca8b4b81d546ac99f0c7de11abf658" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fca8b4b81d546ac99f0c7de11abf658" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33031 } } }
I20260619 02:34:05.278282 6875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fca8b4b81d546ac99f0c7de11abf658 [sys.catalog]: This master's current role is: LEADER
I20260619 02:34:05.278555 6882 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:34:05.282671 6882 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:34:05.289530 6882 catalog_manager.cc:1347] Generated new cluster ID: bddb4544f9df42808a5fd72d6d9e01a0
I20260619 02:34:05.289628 6882 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:34:05.307588 6882 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:34:05.308028 6882 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:34:05.312848 6882 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5fca8b4b81d546ac99f0c7de11abf658: Generated new TSK 0
I20260619 02:34:05.313728 6882 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:34:05.318651 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 6795
2026-06-19T02:34:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1506 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:34:05 dist-test-slave-4qw6 krb5kdc[6894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:34:05 dist-test-slave-4qw6 krb5kdc[6894](info): set up 2 sockets
Jun 19 02:34:05 dist-test-slave-4qw6 krb5kdc[6894](info): commencing operation
krb5kdc: starting...
W20260619 02:34:06.394210 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:34:06 dist-test-slave-4qw6 krb5kdc[6894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:34:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:34:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:34:06.530663 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:40263
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:45605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:40263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:34:06.653445 6907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:34:06.653915 6907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:34:06.654014 6907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:34:06.658565 6907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:34:06.658711 6907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:34:06.658752 6907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:34:06.658792 6907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:34:06.658831 6907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:34:06.665201 6907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:45605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:40263
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:40263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:34:06.666522 6907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:34:06.667644 6907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:34:06.676976 6912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:06.676993 6915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:06.676993 6913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:34:06.677313 6907 server_base.cc:1034] running on GCE node
I20260619 02:34:06.678056 6907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:34:06.679437 6907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:34:06.680684 6907 hybrid_clock.cc:648] HybridClock initialized: now 1781836446680654 us; error 62 us; skew 500 ppm
Jun 19 02:34:06 dist-test-slave-4qw6 krb5kdc[6894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836446, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:34:06.688611 6907 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:34:06.689702 6907 webserver.cc:460] Webserver started at http://127.17.96.254:43617/ using document root <none> and password file <none>
I20260619 02:34:06.690418 6907 fs_manager.cc:359] Metadata directory not provided
I20260619 02:34:06.690501 6907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:34:06.690691 6907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:34:06.693008 6907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "03da73c2c0ad4038b42726cec6df74e7"
format_stamp: "Formatted at 2026-06-19 02:34:06 on dist-test-slave-4qw6"
server_key: "015eda81b670dce08cedaab0dbd19e4e"
server_key_iv: "8dcc59f0e69cd3692ad66074b479d80b"
server_key_version: "encryptionkey@0"
I20260619 02:34:06.693606 6907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "03da73c2c0ad4038b42726cec6df74e7"
format_stamp: "Formatted at 2026-06-19 02:34:06 on dist-test-slave-4qw6"
server_key: "015eda81b670dce08cedaab0dbd19e4e"
server_key_iv: "8dcc59f0e69cd3692ad66074b479d80b"
server_key_version: "encryptionkey@0"
I20260619 02:34:06.698276 6907 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260619 02:34:06.701176 6922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:34:06.702744 6907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:34:06.702922 6907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "03da73c2c0ad4038b42726cec6df74e7"
format_stamp: "Formatted at 2026-06-19 02:34:06 on dist-test-slave-4qw6"
server_key: "015eda81b670dce08cedaab0dbd19e4e"
server_key_iv: "8dcc59f0e69cd3692ad66074b479d80b"
server_key_version: "encryptionkey@0"
I20260619 02:34:06.703076 6907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:34:06.722378 6907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:34:06.725966 6907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:34:06.726208 6907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:34:06.734829 6907 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:40263
I20260619 02:34:06.734856 6974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:40263 every 8 connection(s)
I20260619 02:34:06.735915 6907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:34:06.738376 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 6907
I20260619 02:34:06.738510 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:34:06.738859 17795 external_mini_cluster.cc:1428] Setting key 2b74f0ab9c5af6caa6c7809af1fbb464
I20260619 02:34:06.739215 6975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:34:06.746624 6975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7: Bootstrap starting.
Jun 19 02:34:06 dist-test-slave-4qw6 krb5kdc[6894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:34:06.749425 6975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7: Neither blocks nor log segments found. Creating new log.
I20260619 02:34:06.750399 6975 log.cc:824] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7: Log is configured to *not* fsync() on all Append() calls
I20260619 02:34:06.752715 6975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7: No bootstrap required, opened a new log
I20260619 02:34:06.755334 6978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:06.740396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57735 (local address 127.17.96.254:40263)
0619 02:34:06.741032 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:06.741048 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:06.741087 (+ 39us) server_negotiation.cc:407] Connection header received
0619 02:34:06.742533 (+ 1446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:06.742555 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:06.742961 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:06.743352 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:06.744239 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:06.745273 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:06.746317 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:06.746885 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:06.747094 (+ 209us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:06.749974 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:06.749980 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:06.749996 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:06.750032 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:06.751977 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:06.752589 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:06.752595 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:06.752600 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:06.752723 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:06.753181 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:06.753183 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:06.753184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:06.753624 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:06.753910 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:06.754211 (+ 301us) server_negotiation.cc:299] Negotiation successful
0619 02:34:06.754493 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":174,"threads_started":1}
I20260619 02:34:06.757208 6975 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03da73c2c0ad4038b42726cec6df74e7" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40263 } }
I20260619 02:34:06.757562 6975 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:34:06.757665 6975 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03da73c2c0ad4038b42726cec6df74e7, State: Initialized, Role: FOLLOWER
I20260619 02:34:06.758191 6975 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03da73c2c0ad4038b42726cec6df74e7" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40263 } }
I20260619 02:34:06.758371 6975 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:34:06.758445 6975 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:34:06.758550 6975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:34:06.759065 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:34:06.760027 6975 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03da73c2c0ad4038b42726cec6df74e7" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40263 } }
I20260619 02:34:06.760573 6975 leader_election.cc:302] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03da73c2c0ad4038b42726cec6df74e7; no voters:
I20260619 02:34:06.760996 6975 leader_election.cc:288] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:34:06.761160 6984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:34:06.762066 6984 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7 [term 1 LEADER]: Becoming Leader. State: Replica: 03da73c2c0ad4038b42726cec6df74e7, State: Running, Role: LEADER
I20260619 02:34:06.762475 6984 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03da73c2c0ad4038b42726cec6df74e7" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40263 } }
I20260619 02:34:06.762627 6975 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:34:06.764935 6987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03da73c2c0ad4038b42726cec6df74e7. Latest consensus state: current_term: 1 leader_uuid: "03da73c2c0ad4038b42726cec6df74e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03da73c2c0ad4038b42726cec6df74e7" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40263 } } }
I20260619 02:34:06.764948 6985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03da73c2c0ad4038b42726cec6df74e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03da73c2c0ad4038b42726cec6df74e7" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 40263 } } }
I20260619 02:34:06.765502 6985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7 [sys.catalog]: This master's current role is: LEADER
I20260619 02:34:06.765837 6987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7 [sys.catalog]: This master's current role is: LEADER
W20260619 02:34:06.768366 6999 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:34:06.768483 6999 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:34:06.769856 7000 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:34:06.770951 7000 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:34:06.771997 6978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:06.761693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54401 (local address 127.17.96.254:40263)
0619 02:34:06.761837 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:06.761841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:06.762050 (+ 209us) server_negotiation.cc:407] Connection header received
0619 02:34:06.762258 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:06.762261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:06.762312 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:06.762364 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:06.763428 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:06.764123 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:06.765360 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:06.765716 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:06.765863 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:06.767863 (+ 2000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:06.767867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:06.767870 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:06.767903 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:06.769806 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:06.770487 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:06.770491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:06.770493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:06.770582 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:06.771006 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:06.771009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:06.771011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:06.771249 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:06.771413 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:06.771735 (+ 322us) server_negotiation.cc:299] Negotiation successful
0619 02:34:06.771840 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 02:34:06.777724 7000 catalog_manager.cc:1347] Generated new cluster ID: f9ec216896ec437b9a04d10523b25c7f
I20260619 02:34:06.777853 7000 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:34:06.806890 7000 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:34:06.808082 7000 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:34:06.813596 7000 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03da73c2c0ad4038b42726cec6df74e7: Generated new TSK 0
I20260619 02:34:06.814489 7000 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:34:06.818508 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 6907
W20260619 02:34:06.824172 6892 connection.cc:535] client connection to 127.17.96.254:40263 recv error: Network error: failed to read from TLS socket (remote: 127.17.96.254:40263): Connection reset by peer (error 104)
2026-06-19T02:34:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1500 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:34:06 dist-test-slave-4qw6 krb5kdc[7006](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:34:06 dist-test-slave-4qw6 krb5kdc[7006](info): set up 2 sockets
Jun 19 02:34:06 dist-test-slave-4qw6 krb5kdc[7006](info): commencing operation
krb5kdc: starting...
W20260619 02:34:07.878449 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:34:07 dist-test-slave-4qw6 krb5kdc[7006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:34:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:34:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:34:08.013218 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:46617
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:41673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:46617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:34:08.129145 7019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:34:08.129565 7019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:34:08.129616 7019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:34:08.133828 7019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:34:08.133912 7019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:34:08.133932 7019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:34:08.133950 7019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:34:08.133967 7019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:34:08.140205 7019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:41673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:46617
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:46617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:34:08.141366 7019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:34:08.142441 7019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:34:08.151612 7025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:08.151626 7024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:08.151630 7027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:34:08.151937 7019 server_base.cc:1034] running on GCE node
I20260619 02:34:08.152593 7019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:34:08.153955 7019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:34:08.155165 7019 hybrid_clock.cc:648] HybridClock initialized: now 1781836448155147 us; error 35 us; skew 500 ppm
Jun 19 02:34:08 dist-test-slave-4qw6 krb5kdc[7006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836448, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:34:08.158355 7019 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:34:08.159353 7019 webserver.cc:460] Webserver started at http://127.17.96.254:40229/ using document root <none> and password file <none>
I20260619 02:34:08.160039 7019 fs_manager.cc:359] Metadata directory not provided
I20260619 02:34:08.160115 7019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:34:08.160302 7019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:34:08.162685 7019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "83e565bb2c5a424089ecce167749be49"
format_stamp: "Formatted at 2026-06-19 02:34:08 on dist-test-slave-4qw6"
server_key: "1bd054360c866d531c3ba0598df79ce8"
server_key_iv: "dfd532051cb207e25fc9455c236c514c"
server_key_version: "encryptionkey@0"
I20260619 02:34:08.163267 7019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "83e565bb2c5a424089ecce167749be49"
format_stamp: "Formatted at 2026-06-19 02:34:08 on dist-test-slave-4qw6"
server_key: "1bd054360c866d531c3ba0598df79ce8"
server_key_iv: "dfd532051cb207e25fc9455c236c514c"
server_key_version: "encryptionkey@0"
I20260619 02:34:08.167464 7019 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:34:08.170238 7034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:34:08.171670 7019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:34:08.171837 7019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "83e565bb2c5a424089ecce167749be49"
format_stamp: "Formatted at 2026-06-19 02:34:08 on dist-test-slave-4qw6"
server_key: "1bd054360c866d531c3ba0598df79ce8"
server_key_iv: "dfd532051cb207e25fc9455c236c514c"
server_key_version: "encryptionkey@0"
I20260619 02:34:08.171976 7019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:34:08.196188 7019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:34:08.199579 7019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:34:08.199821 7019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:34:08.207887 7019 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:46617
I20260619 02:34:08.207911 7086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:46617 every 8 connection(s)
I20260619 02:34:08.209210 7019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:34:08.210860 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 7019
I20260619 02:34:08.211071 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:34:08.211385 17795 external_mini_cluster.cc:1428] Setting key 31fa7e1c26ac477936118a73a7ddb6c2
I20260619 02:34:08.213641 7087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:34:08.220939 7087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49: Bootstrap starting.
I20260619 02:34:08.221485 7090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:08.212891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57743 (local address 127.17.96.254:46617)
0619 02:34:08.213357 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:08.213377 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:08.213430 (+ 53us) server_negotiation.cc:407] Connection header received
0619 02:34:08.214222 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:08.214243 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:08.214618 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:08.214939 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:08.215402 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:08.216443 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:08.217705 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:08.218311 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:08.218441 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:08.219492 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:08.219498 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:08.219512 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:34:08.219973 (+ 461us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:08.220033 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:34:08.220057 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:08.220162 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:08.220395 (+ 233us) server_negotiation.cc:299] Negotiation successful
0619 02:34:08.220585 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":173,"threads_started":1}
I20260619 02:34:08.223457 7087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49: Neither blocks nor log segments found. Creating new log.
I20260619 02:34:08.224234 7087 log.cc:824] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49: Log is configured to *not* fsync() on all Append() calls
I20260619 02:34:08.224741 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:34:08.226562 7087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49: No bootstrap required, opened a new log
I20260619 02:34:08.230487 7087 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83e565bb2c5a424089ecce167749be49" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46617 } }
I20260619 02:34:08.230782 7087 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:34:08.230870 7087 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83e565bb2c5a424089ecce167749be49, State: Initialized, Role: FOLLOWER
I20260619 02:34:08.231096 7090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:08.226895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40313 (local address 127.17.96.254:46617)
0619 02:34:08.227031 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:08.227036 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:08.227261 (+ 225us) server_negotiation.cc:407] Connection header received
0619 02:34:08.227421 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:08.227425 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:08.227468 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:08.227572 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:08.227848 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:08.228390 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:08.229257 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:08.229497 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:08.229620 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:08.230561 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:08.230565 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:08.230567 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:34:08.230581 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:08.230615 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:34:08.230626 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:08.230728 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:08.230899 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 02:34:08.230949 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:34:08.231406 7087 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83e565bb2c5a424089ecce167749be49" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46617 } }
I20260619 02:34:08.231590 7087 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:34:08.231662 7087 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:34:08.231766 7087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:34:08.232972 7087 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83e565bb2c5a424089ecce167749be49" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46617 } }
I20260619 02:34:08.233305 7087 leader_election.cc:302] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83e565bb2c5a424089ecce167749be49; no voters:
I20260619 02:34:08.233599 7087 leader_election.cc:288] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:34:08.233785 7097 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:34:08.234047 7097 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49 [term 1 LEADER]: Becoming Leader. State: Replica: 83e565bb2c5a424089ecce167749be49, State: Running, Role: LEADER
I20260619 02:34:08.234434 7097 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83e565bb2c5a424089ecce167749be49" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46617 } }
I20260619 02:34:08.235028 7087 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:34:08.236164 7099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83e565bb2c5a424089ecce167749be49. Latest consensus state: current_term: 1 leader_uuid: "83e565bb2c5a424089ecce167749be49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83e565bb2c5a424089ecce167749be49" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46617 } } }
I20260619 02:34:08.236311 7099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49 [sys.catalog]: This master's current role is: LEADER
I20260619 02:34:08.236725 7098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83e565bb2c5a424089ecce167749be49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83e565bb2c5a424089ecce167749be49" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46617 } } }
I20260619 02:34:08.237429 7098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49 [sys.catalog]: This master's current role is: LEADER
W20260619 02:34:08.240264 7113 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:34:08.240378 7113 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:34:08.240482 7106 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:34:08.241268 7106 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:34:08.247763 7106 catalog_manager.cc:1347] Generated new cluster ID: 875d5ae84fb846dabcbe6f30ec895b78
I20260619 02:34:08.247865 7106 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:34:08.257419 7106 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:34:08.258519 7106 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:34:08.265956 7106 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83e565bb2c5a424089ecce167749be49: Generated new TSK 0
I20260619 02:34:08.266777 7106 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:34:08.272749 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 7019
W20260619 02:34:08.278429 7004 connection.cc:535] client connection to 127.17.96.254:46617 recv error: Network error: failed to read from TLS socket (remote: 127.17.96.254:46617): Connection reset by peer (error 104)
2026-06-19T02:34:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1454 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:34:08 dist-test-slave-4qw6 krb5kdc[7118](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:34:08 dist-test-slave-4qw6 krb5kdc[7118](info): set up 2 sockets
Jun 19 02:34:08 dist-test-slave-4qw6 krb5kdc[7118](info): commencing operation
krb5kdc: starting...
W20260619 02:34:09.322124 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:34:09 dist-test-slave-4qw6 krb5kdc[7118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:34:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:34:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:34:09.449551 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:34691
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:46205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:34691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:34:09.566843 7131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:34:09.567227 7131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:34:09.567312 7131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:34:09.571746 7131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:34:09.571916 7131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:34:09.571959 7131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:34:09.571996 7131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:34:09.572028 7131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:34:09.577845 7131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:46205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:34691
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:34691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:34:09.579078 7131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:34:09.580156 7131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:34:09.589229 7137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:09.589229 7139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:34:09.589360 7131 server_base.cc:1034] running on GCE node
W20260619 02:34:09.589200 7136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:34:09.590276 7131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:34:09.591421 7131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:34:09.592600 7131 hybrid_clock.cc:648] HybridClock initialized: now 1781836449592572 us; error 43 us; skew 500 ppm
Jun 19 02:34:09 dist-test-slave-4qw6 krb5kdc[7118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836449, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:34:09.595625 7131 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:34:09.596647 7131 webserver.cc:460] Webserver started at http://127.17.96.254:33091/ using document root <none> and password file <none>
I20260619 02:34:09.597188 7131 fs_manager.cc:359] Metadata directory not provided
I20260619 02:34:09.597249 7131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:34:09.597430 7131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:34:09.599629 7131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "3ed633f3712c4003a942e16eb2bdb49d"
format_stamp: "Formatted at 2026-06-19 02:34:09 on dist-test-slave-4qw6"
server_key: "e96b5217b547918c3f1b7206cc7c8bca"
server_key_iv: "c61f687652949e9b3a680b387689a43f"
server_key_version: "encryptionkey@0"
I20260619 02:34:09.600178 7131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "3ed633f3712c4003a942e16eb2bdb49d"
format_stamp: "Formatted at 2026-06-19 02:34:09 on dist-test-slave-4qw6"
server_key: "e96b5217b547918c3f1b7206cc7c8bca"
server_key_iv: "c61f687652949e9b3a680b387689a43f"
server_key_version: "encryptionkey@0"
I20260619 02:34:09.604162 7131 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260619 02:34:09.606802 7146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:34:09.608142 7131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:34:09.608335 7131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "3ed633f3712c4003a942e16eb2bdb49d"
format_stamp: "Formatted at 2026-06-19 02:34:09 on dist-test-slave-4qw6"
server_key: "e96b5217b547918c3f1b7206cc7c8bca"
server_key_iv: "c61f687652949e9b3a680b387689a43f"
server_key_version: "encryptionkey@0"
I20260619 02:34:09.608484 7131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:34:09.613324 7131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:34:09.616592 7131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:34:09.616815 7131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:34:09.624432 7131 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:34691
I20260619 02:34:09.624451 7198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:34691 every 8 connection(s)
I20260619 02:34:09.625473 7131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:34:09.626717 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 7131
I20260619 02:34:09.626883 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:34:09.627162 17795 external_mini_cluster.cc:1428] Setting key c341783d9f6dbba61531582ce656a1e0
I20260619 02:34:09.628777 7199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:34:09.632776 7202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:09.628693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38863 (local address 127.17.96.254:34691)
0619 02:34:09.629172 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:09.629188 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:09.629225 (+ 37us) server_negotiation.cc:407] Connection header received
0619 02:34:09.629992 (+ 767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:09.630011 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:09.630310 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:09.630680 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:09.631549 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:09.631554 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:09.631566 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:34:09.631577 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:09.631632 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:34:09.631651 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:09.631812 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:09.631895 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 02:34:09.632058 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":155,"threads_started":1}
I20260619 02:34:09.635597 7199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed633f3712c4003a942e16eb2bdb49d: Bootstrap starting.
I20260619 02:34:09.636056 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:34:09.638252 7199 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ed633f3712c4003a942e16eb2bdb49d: Neither blocks nor log segments found. Creating new log.
I20260619 02:34:09.639246 7199 log.cc:824] T 00000000000000000000000000000000 P 3ed633f3712c4003a942e16eb2bdb49d: Log is configured to *not* fsync() on all Append() calls
I20260619 02:34:09.640204 7202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:09.638293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34751 (local address 127.17.96.254:34691)
0619 02:34:09.638445 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:09.638449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:09.638662 (+ 213us) server_negotiation.cc:407] Connection header received
0619 02:34:09.638748 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:09.638751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:09.638795 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:09.638890 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:09.639674 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:09.639680 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:09.639683 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:34:09.639698 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:09.639728 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:34:09.639746 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:09.639840 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:09.639996 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 02:34:09.640047 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 02:34:09.641727 7199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed633f3712c4003a942e16eb2bdb49d: No bootstrap required, opened a new log
I20260619 02:34:09.645468 7199 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ed633f3712c4003a942e16eb2bdb49d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed633f3712c4003a942e16eb2bdb49d" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34691 } }
I20260619 02:34:09.645838 7199 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ed633f3712c4003a942e16eb2bdb49d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:34:09.645913 7199 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ed633f3712c4003a942e16eb2bdb49d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed633f3712c4003a942e16eb2bdb49d, State: Initialized, Role: FOLLOWER
I20260619 02:34:09.646421 7199 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ed633f3712c4003a942e16eb2bdb49d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed633f3712c4003a942e16eb2bdb49d" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34691 } }
I20260619 02:34:09.646612 7199 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ed633f3712c4003a942e16eb2bdb49d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:34:09.646692 7199 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ed633f3712c4003a942e16eb2bdb49d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:34:09.646803 7199 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ed633f3712c4003a942e16eb2bdb49d [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:34:09.648063 7199 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ed633f3712c4003a942e16eb2bdb49d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed633f3712c4003a942e16eb2bdb49d" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34691 } }
I20260619 02:34:09.648470 7199 leader_election.cc:302] T 00000000000000000000000000000000 P 3ed633f3712c4003a942e16eb2bdb49d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ed633f3712c4003a942e16eb2bdb49d; no voters:
I20260619 02:34:09.648865 7199 leader_election.cc:288] T 00000000000000000000000000000000 P 3ed633f3712c4003a942e16eb2bdb49d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:34:09.649094 7209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ed633f3712c4003a942e16eb2bdb49d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:34:09.649422 7209 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ed633f3712c4003a942e16eb2bdb49d [term 1 LEADER]: Becoming Leader. State: Replica: 3ed633f3712c4003a942e16eb2bdb49d, State: Running, Role: LEADER
I20260619 02:34:09.649871 7209 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ed633f3712c4003a942e16eb2bdb49d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed633f3712c4003a942e16eb2bdb49d" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34691 } }
I20260619 02:34:09.650357 7199 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ed633f3712c4003a942e16eb2bdb49d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:34:09.651659 7211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed633f3712c4003a942e16eb2bdb49d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ed633f3712c4003a942e16eb2bdb49d. Latest consensus state: current_term: 1 leader_uuid: "3ed633f3712c4003a942e16eb2bdb49d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed633f3712c4003a942e16eb2bdb49d" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34691 } } }
I20260619 02:34:09.651825 7211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed633f3712c4003a942e16eb2bdb49d [sys.catalog]: This master's current role is: LEADER
I20260619 02:34:09.652225 7218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:34:09.652575 7210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed633f3712c4003a942e16eb2bdb49d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ed633f3712c4003a942e16eb2bdb49d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed633f3712c4003a942e16eb2bdb49d" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34691 } } }
I20260619 02:34:09.652714 7210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed633f3712c4003a942e16eb2bdb49d [sys.catalog]: This master's current role is: LEADER
I20260619 02:34:09.655345 7218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:34:09.661859 7218 catalog_manager.cc:1347] Generated new cluster ID: ed0ad6c6261547e8acf9681e5a1707ee
I20260619 02:34:09.661976 7218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:34:09.676216 7218 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:34:09.676621 7218 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:34:09.692029 7218 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ed633f3712c4003a942e16eb2bdb49d: Generated new TSK 0
I20260619 02:34:09.692905 7218 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:34:09.696548 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 7131
2026-06-19T02:34:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1423 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:34:09 dist-test-slave-4qw6 krb5kdc[7230](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:34:09 dist-test-slave-4qw6 krb5kdc[7230](info): set up 2 sockets
Jun 19 02:34:09 dist-test-slave-4qw6 krb5kdc[7230](info): commencing operation
krb5kdc: starting...
W20260619 02:34:10.746243 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:34:10 dist-test-slave-4qw6 krb5kdc[7230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:34:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:34:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:34:10.883034 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:44913
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:35497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:44913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:34:11.005745 7243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:34:11.006103 7243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:34:11.006150 7243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:34:11.010727 7243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:34:11.010804 7243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:34:11.010825 7243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:34:11.010845 7243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:34:11.010861 7243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:34:11.016866 7243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:35497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:44913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:44913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:34:11.018213 7243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:34:11.019340 7243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:34:11.028489 7248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:34:11.028746 7243 server_base.cc:1034] running on GCE node
W20260619 02:34:11.028484 7249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:11.028535 7251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:34:11.029580 7243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:34:11.030944 7243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:34:11.032171 7243 hybrid_clock.cc:648] HybridClock initialized: now 1781836451032152 us; error 36 us; skew 500 ppm
Jun 19 02:34:11 dist-test-slave-4qw6 krb5kdc[7230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836451, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:34:11.035269 7243 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:34:11.036250 7243 webserver.cc:460] Webserver started at http://127.17.96.254:36127/ using document root <none> and password file <none>
I20260619 02:34:11.036834 7243 fs_manager.cc:359] Metadata directory not provided
I20260619 02:34:11.036902 7243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:34:11.037079 7243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:34:11.039381 7243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "8ade9105a52244759b4ea7802ffd1627"
format_stamp: "Formatted at 2026-06-19 02:34:11 on dist-test-slave-4qw6"
server_key: "3d7a83a6da09629bfa205a2c7bcafc1f"
server_key_iv: "358621bd655e544d5c37532faa58b61a"
server_key_version: "encryptionkey@0"
I20260619 02:34:11.039963 7243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "8ade9105a52244759b4ea7802ffd1627"
format_stamp: "Formatted at 2026-06-19 02:34:11 on dist-test-slave-4qw6"
server_key: "3d7a83a6da09629bfa205a2c7bcafc1f"
server_key_iv: "358621bd655e544d5c37532faa58b61a"
server_key_version: "encryptionkey@0"
I20260619 02:34:11.044252 7243 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:34:11.047168 7258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:34:11.048619 7243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:34:11.048749 7243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "8ade9105a52244759b4ea7802ffd1627"
format_stamp: "Formatted at 2026-06-19 02:34:11 on dist-test-slave-4qw6"
server_key: "3d7a83a6da09629bfa205a2c7bcafc1f"
server_key_iv: "358621bd655e544d5c37532faa58b61a"
server_key_version: "encryptionkey@0"
I20260619 02:34:11.048954 7243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:34:11.083149 7243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:34:11.086584 7243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:34:11.086846 7243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:34:11.094682 7243 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:44913
I20260619 02:34:11.094693 7310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:44913 every 8 connection(s)
I20260619 02:34:11.095732 7243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:34:11.098834 7311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:34:11.101415 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 7243
I20260619 02:34:11.101581 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:34:11.101904 17795 external_mini_cluster.cc:1428] Setting key 1750a98cf02348b1d00a700651e0d635
I20260619 02:34:11.105710 7311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ade9105a52244759b4ea7802ffd1627: Bootstrap starting.
I20260619 02:34:11.108228 7311 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ade9105a52244759b4ea7802ffd1627: Neither blocks nor log segments found. Creating new log.
I20260619 02:34:11.109081 7311 log.cc:824] T 00000000000000000000000000000000 P 8ade9105a52244759b4ea7802ffd1627: Log is configured to *not* fsync() on all Append() calls
Jun 19 02:34:11 dist-test-slave-4qw6 krb5kdc[7230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:34:11.111686 7311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ade9105a52244759b4ea7802ffd1627: No bootstrap required, opened a new log
I20260619 02:34:11.115923 7311 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ade9105a52244759b4ea7802ffd1627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ade9105a52244759b4ea7802ffd1627" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44913 } }
I20260619 02:34:11.116300 7311 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ade9105a52244759b4ea7802ffd1627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:34:11.116425 7311 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ade9105a52244759b4ea7802ffd1627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ade9105a52244759b4ea7802ffd1627, State: Initialized, Role: FOLLOWER
I20260619 02:34:11.117009 7311 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ade9105a52244759b4ea7802ffd1627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ade9105a52244759b4ea7802ffd1627" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44913 } }
I20260619 02:34:11.117221 7311 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ade9105a52244759b4ea7802ffd1627 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:34:11.117333 7311 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ade9105a52244759b4ea7802ffd1627 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:34:11.117456 7311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ade9105a52244759b4ea7802ffd1627 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:34:11.118002 7314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:11.103536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48587 (local address 127.17.96.254:44913)
0619 02:34:11.104018 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:11.104038 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:11.104075 (+ 37us) server_negotiation.cc:407] Connection header received
0619 02:34:11.104753 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:11.104771 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:11.105135 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:11.105502 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:11.106400 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:11.107411 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:11.108340 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:11.108923 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:11.109133 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:11.112207 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:11.112214 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:11.112230 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:11.112263 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:11.114305 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:11.114891 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:11.114904 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:11.114911 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:11.115042 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:11.115377 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:11.115382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:11.115384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:11.115860 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:11.116112 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:11.116446 (+ 334us) server_negotiation.cc:299] Negotiation successful
0619 02:34:11.116831 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":161,"threads_started":1}
I20260619 02:34:11.118795 7311 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ade9105a52244759b4ea7802ffd1627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ade9105a52244759b4ea7802ffd1627" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44913 } }
I20260619 02:34:11.119262 7311 leader_election.cc:302] T 00000000000000000000000000000000 P 8ade9105a52244759b4ea7802ffd1627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ade9105a52244759b4ea7802ffd1627; no voters:
I20260619 02:34:11.119678 7311 leader_election.cc:288] T 00000000000000000000000000000000 P 8ade9105a52244759b4ea7802ffd1627 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:34:11.119834 7316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ade9105a52244759b4ea7802ffd1627 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:34:11.120119 7316 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ade9105a52244759b4ea7802ffd1627 [term 1 LEADER]: Becoming Leader. State: Replica: 8ade9105a52244759b4ea7802ffd1627, State: Running, Role: LEADER
I20260619 02:34:11.120503 7316 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ade9105a52244759b4ea7802ffd1627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ade9105a52244759b4ea7802ffd1627" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44913 } }
I20260619 02:34:11.121160 7311 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ade9105a52244759b4ea7802ffd1627 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:34:11.121513 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:34:11.122372 7317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ade9105a52244759b4ea7802ffd1627 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ade9105a52244759b4ea7802ffd1627" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ade9105a52244759b4ea7802ffd1627" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44913 } } }
I20260619 02:34:11.122589 7317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ade9105a52244759b4ea7802ffd1627 [sys.catalog]: This master's current role is: LEADER
I20260619 02:34:11.122882 7317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ade9105a52244759b4ea7802ffd1627 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ade9105a52244759b4ea7802ffd1627. Latest consensus state: current_term: 1 leader_uuid: "8ade9105a52244759b4ea7802ffd1627" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ade9105a52244759b4ea7802ffd1627" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44913 } } }
I20260619 02:34:11.123036 7317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ade9105a52244759b4ea7802ffd1627 [sys.catalog]: This master's current role is: LEADER
I20260619 02:34:11.123952 7325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:34:11.127607 7325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:34:11.134083 7314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:11.124276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37345 (local address 127.17.96.254:44913)
0619 02:34:11.124452 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:11.124457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:11.124644 (+ 187us) server_negotiation.cc:407] Connection header received
0619 02:34:11.124806 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:11.124808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:11.124852 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:11.124927 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:11.126032 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:11.126817 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:11.127962 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:11.128310 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:11.128427 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:11.130448 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:11.130452 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:11.130466 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:11.130506 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:11.132062 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:11.132552 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:11.132558 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:11.132564 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:11.132643 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:11.132961 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:11.132968 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:11.132973 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:11.133227 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:11.133361 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:11.133726 (+ 365us) server_negotiation.cc:299] Negotiation successful
0619 02:34:11.133908 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 02:34:11.135789 7325 catalog_manager.cc:1347] Generated new cluster ID: 510d32661a64485ca8077070b04bbbdb
I20260619 02:34:11.135867 7325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:34:11.159660 7325 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:34:11.160998 7325 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:34:11.176541 7325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ade9105a52244759b4ea7802ffd1627: Generated new TSK 0
I20260619 02:34:11.177394 7325 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:34:11.184075 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 7243
2026-06-19T02:34:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1487 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:34:11 dist-test-slave-4qw6 krb5kdc[7342](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:34:11 dist-test-slave-4qw6 krb5kdc[7342](info): set up 2 sockets
Jun 19 02:34:11 dist-test-slave-4qw6 krb5kdc[7342](info): commencing operation
krb5kdc: starting...
W20260619 02:34:12.238112 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:34:12 dist-test-slave-4qw6 krb5kdc[7342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:34:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:34:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:34:12.369004 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:34749
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:35967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:34749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:34:12.490156 7355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:34:12.490556 7355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:34:12.490660 7355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:34:12.495023 7355 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:34:12.495153 7355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:34:12.495198 7355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:34:12.495247 7355 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:34:12.495285 7355 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:34:12.501067 7355 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:35967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:34749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:34749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:34:12.502453 7355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:34:12.503648 7355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:34:12.513309 7363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:12.513274 7360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:12.513286 7361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:34:12.513584 7355 server_base.cc:1034] running on GCE node
I20260619 02:34:12.514278 7355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:34:12.515571 7355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:34:12.516793 7355 hybrid_clock.cc:648] HybridClock initialized: now 1781836452516775 us; error 36 us; skew 500 ppm
Jun 19 02:34:12 dist-test-slave-4qw6 krb5kdc[7342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836452, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:34:12.523618 7355 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:34:12.524809 7355 webserver.cc:460] Webserver started at http://127.17.96.254:41281/ using document root <none> and password file <none>
I20260619 02:34:12.525493 7355 fs_manager.cc:359] Metadata directory not provided
I20260619 02:34:12.525605 7355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:34:12.525947 7355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:34:12.528242 7355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "a94c0e6bd80e4ca892841abfd0cc8813"
format_stamp: "Formatted at 2026-06-19 02:34:12 on dist-test-slave-4qw6"
server_key: "ab69da2683ff49fc1723a4da0eba4b89"
server_key_iv: "66490cc39041e24443c8f98c9d5a2259"
server_key_version: "encryptionkey@0"
I20260619 02:34:12.528950 7355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "a94c0e6bd80e4ca892841abfd0cc8813"
format_stamp: "Formatted at 2026-06-19 02:34:12 on dist-test-slave-4qw6"
server_key: "ab69da2683ff49fc1723a4da0eba4b89"
server_key_iv: "66490cc39041e24443c8f98c9d5a2259"
server_key_version: "encryptionkey@0"
I20260619 02:34:12.533196 7355 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 02:34:12.536082 7370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:34:12.537521 7355 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:34:12.537726 7355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "a94c0e6bd80e4ca892841abfd0cc8813"
format_stamp: "Formatted at 2026-06-19 02:34:12 on dist-test-slave-4qw6"
server_key: "ab69da2683ff49fc1723a4da0eba4b89"
server_key_iv: "66490cc39041e24443c8f98c9d5a2259"
server_key_version: "encryptionkey@0"
I20260619 02:34:12.537963 7355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:34:12.554896 7355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:34:12.558295 7355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:34:12.558578 7355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:34:12.567103 7355 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:34749
I20260619 02:34:12.567116 7422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:34749 every 8 connection(s)
I20260619 02:34:12.568236 7355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:34:12.571398 7423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:34:12.576548 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 7355
I20260619 02:34:12.576704 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:34:12.576997 17795 external_mini_cluster.cc:1428] Setting key 8143f00ca9d563d63d098ef0249061a3
I20260619 02:34:12.578349 7423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a94c0e6bd80e4ca892841abfd0cc8813: Bootstrap starting.
I20260619 02:34:12.580873 7423 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a94c0e6bd80e4ca892841abfd0cc8813: Neither blocks nor log segments found. Creating new log.
I20260619 02:34:12.581736 7423 log.cc:824] T 00000000000000000000000000000000 P a94c0e6bd80e4ca892841abfd0cc8813: Log is configured to *not* fsync() on all Append() calls
I20260619 02:34:12.584283 7423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a94c0e6bd80e4ca892841abfd0cc8813: No bootstrap required, opened a new log
I20260619 02:34:12.587584 7426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:12.578697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36221 (local address 127.17.96.254:34749)
0619 02:34:12.579198 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:12.579225 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:12.579265 (+ 40us) server_negotiation.cc:407] Connection header received
0619 02:34:12.579983 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:12.580001 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:12.580373 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:12.580686 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:12.581149 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:12.582369 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:12.583423 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:12.584142 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:12.584329 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:12.585246 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:12.585253 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:12.585270 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:34:12.585876 (+ 606us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:12.585954 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:34:12.585991 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:12.586099 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:12.586252 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 02:34:12.586485 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":174,"threads_started":1}
I20260619 02:34:12.589195 7423 raft_consensus.cc:348] T 00000000000000000000000000000000 P a94c0e6bd80e4ca892841abfd0cc8813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94c0e6bd80e4ca892841abfd0cc8813" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34749 } }
I20260619 02:34:12.589519 7423 raft_consensus.cc:374] T 00000000000000000000000000000000 P a94c0e6bd80e4ca892841abfd0cc8813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:34:12.589576 7423 raft_consensus.cc:729] T 00000000000000000000000000000000 P a94c0e6bd80e4ca892841abfd0cc8813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a94c0e6bd80e4ca892841abfd0cc8813, State: Initialized, Role: FOLLOWER
I20260619 02:34:12.590180 7423 consensus_queue.cc:260] T 00000000000000000000000000000000 P a94c0e6bd80e4ca892841abfd0cc8813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94c0e6bd80e4ca892841abfd0cc8813" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34749 } }
I20260619 02:34:12.590387 7423 raft_consensus.cc:388] T 00000000000000000000000000000000 P a94c0e6bd80e4ca892841abfd0cc8813 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:34:12.590500 7423 raft_consensus.cc:482] T 00000000000000000000000000000000 P a94c0e6bd80e4ca892841abfd0cc8813 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:34:12.590616 7423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a94c0e6bd80e4ca892841abfd0cc8813 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:34:12.591041 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:34:12.592087 7423 raft_consensus.cc:504] T 00000000000000000000000000000000 P a94c0e6bd80e4ca892841abfd0cc8813 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94c0e6bd80e4ca892841abfd0cc8813" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34749 } }
I20260619 02:34:12.592541 7423 leader_election.cc:302] T 00000000000000000000000000000000 P a94c0e6bd80e4ca892841abfd0cc8813 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a94c0e6bd80e4ca892841abfd0cc8813; no voters:
I20260619 02:34:12.592936 7423 leader_election.cc:288] T 00000000000000000000000000000000 P a94c0e6bd80e4ca892841abfd0cc8813 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:34:12.593056 7432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a94c0e6bd80e4ca892841abfd0cc8813 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:34:12.593294 7432 raft_consensus.cc:686] T 00000000000000000000000000000000 P a94c0e6bd80e4ca892841abfd0cc8813 [term 1 LEADER]: Becoming Leader. State: Replica: a94c0e6bd80e4ca892841abfd0cc8813, State: Running, Role: LEADER
I20260619 02:34:12.593698 7432 consensus_queue.cc:237] T 00000000000000000000000000000000 P a94c0e6bd80e4ca892841abfd0cc8813 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94c0e6bd80e4ca892841abfd0cc8813" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34749 } }
I20260619 02:34:12.594576 7423 sys_catalog.cc:564] T 00000000000000000000000000000000 P a94c0e6bd80e4ca892841abfd0cc8813 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:34:12.595559 7433 sys_catalog.cc:455] T 00000000000000000000000000000000 P a94c0e6bd80e4ca892841abfd0cc8813 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a94c0e6bd80e4ca892841abfd0cc8813" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94c0e6bd80e4ca892841abfd0cc8813" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34749 } } }
I20260619 02:34:12.595718 7433 sys_catalog.cc:458] T 00000000000000000000000000000000 P a94c0e6bd80e4ca892841abfd0cc8813 [sys.catalog]: This master's current role is: LEADER
I20260619 02:34:12.595970 7433 sys_catalog.cc:455] T 00000000000000000000000000000000 P a94c0e6bd80e4ca892841abfd0cc8813 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a94c0e6bd80e4ca892841abfd0cc8813. Latest consensus state: current_term: 1 leader_uuid: "a94c0e6bd80e4ca892841abfd0cc8813" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94c0e6bd80e4ca892841abfd0cc8813" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 34749 } } }
I20260619 02:34:12.596081 7433 sys_catalog.cc:458] T 00000000000000000000000000000000 P a94c0e6bd80e4ca892841abfd0cc8813 [sys.catalog]: This master's current role is: LEADER
I20260619 02:34:12.596403 7442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:34:12.600421 7442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:34:12.604384 7426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:12.593616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48855 (local address 127.17.96.254:34749)
0619 02:34:12.593734 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:12.593739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:12.598244 (+ 4505us) server_negotiation.cc:407] Connection header received
0619 02:34:12.598477 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:12.598480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:12.598562 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:12.598689 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:12.599060 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:12.599881 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:12.602140 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:12.602478 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:12.602595 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:12.603694 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:12.603698 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:12.603701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:34:12.603717 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:12.603748 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:34:12.603762 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:12.603824 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:12.604159 (+ 335us) server_negotiation.cc:299] Negotiation successful
0619 02:34:12.604240 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:34:12.607764 7442 catalog_manager.cc:1347] Generated new cluster ID: 927190d7be7b4197807bdec79980edab
I20260619 02:34:12.607859 7442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:34:12.626734 7442 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:34:12.627923 7442 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:34:12.639976 7442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a94c0e6bd80e4ca892841abfd0cc8813: Generated new TSK 0
I20260619 02:34:12.640780 7442 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:34:12.655040 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 7355
2026-06-19T02:34:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1471 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:34:12 dist-test-slave-4qw6 krb5kdc[7454](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:34:12 dist-test-slave-4qw6 krb5kdc[7454](info): set up 2 sockets
Jun 19 02:34:12 dist-test-slave-4qw6 krb5kdc[7454](info): commencing operation
krb5kdc: starting...
W20260619 02:34:13.699007 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:34:13 dist-test-slave-4qw6 krb5kdc[7454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:34:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:34:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:34:13.829741 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:43695
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:37841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:43695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:34:13.950031 7467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:34:13.950438 7467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:34:13.950534 7467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:34:13.954835 7467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:34:13.954988 7467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:34:13.955040 7467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:34:13.955075 7467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:34:13.955113 7467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:34:13.961035 7467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:37841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:43695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:43695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:34:13.962350 7467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:34:13.963481 7467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:34:13.972890 7472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:34:13.972892 7467 server_base.cc:1034] running on GCE node
W20260619 02:34:13.972890 7475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:13.972892 7473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:34:13.973901 7467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:34:13.975135 7467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:34:13.976521 7467 hybrid_clock.cc:648] HybridClock initialized: now 1781836453976501 us; error 38 us; skew 500 ppm
Jun 19 02:34:13 dist-test-slave-4qw6 krb5kdc[7454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836453, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:34:13.979645 7467 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:34:13.980824 7467 webserver.cc:460] Webserver started at http://127.17.96.254:46803/ using document root <none> and password file <none>
I20260619 02:34:13.981475 7467 fs_manager.cc:359] Metadata directory not provided
I20260619 02:34:13.981593 7467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:34:13.981921 7467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:34:13.984284 7467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "3cca425719b24cecbe3fd8bf308655e8"
format_stamp: "Formatted at 2026-06-19 02:34:13 on dist-test-slave-4qw6"
server_key: "15c691a3d44b56a70d295c842f2f9845"
server_key_iv: "889e37f0cbb7c591a7cf3980f278ce67"
server_key_version: "encryptionkey@0"
I20260619 02:34:13.985066 7467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "3cca425719b24cecbe3fd8bf308655e8"
format_stamp: "Formatted at 2026-06-19 02:34:13 on dist-test-slave-4qw6"
server_key: "15c691a3d44b56a70d295c842f2f9845"
server_key_iv: "889e37f0cbb7c591a7cf3980f278ce67"
server_key_version: "encryptionkey@0"
I20260619 02:34:13.989442 7467 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 02:34:13.992308 7482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:34:13.993683 7467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:34:13.993913 7467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "3cca425719b24cecbe3fd8bf308655e8"
format_stamp: "Formatted at 2026-06-19 02:34:13 on dist-test-slave-4qw6"
server_key: "15c691a3d44b56a70d295c842f2f9845"
server_key_iv: "889e37f0cbb7c591a7cf3980f278ce67"
server_key_version: "encryptionkey@0"
I20260619 02:34:13.994097 7467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:34:13.999372 7467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:34:14.002710 7467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:34:14.002935 7467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:34:14.010597 7467 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:43695
I20260619 02:34:14.010653 7534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:43695 every 8 connection(s)
I20260619 02:34:14.011624 7467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:34:14.014653 7535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:34:14.017272 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 7467
I20260619 02:34:14.017418 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:34:14.017742 17795 external_mini_cluster.cc:1428] Setting key 3fecbb89fe617c8d270376ae0505b26f
I20260619 02:34:14.021462 7535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cca425719b24cecbe3fd8bf308655e8: Bootstrap starting.
I20260619 02:34:14.023532 7538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:14.019363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58221 (local address 127.17.96.254:43695)
0619 02:34:14.019880 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:14.019896 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:14.019935 (+ 39us) server_negotiation.cc:407] Connection header received
0619 02:34:14.020618 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:14.020636 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:14.020939 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:14.021309 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:14.022189 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:14.022194 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:14.022206 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:34:14.022216 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:14.022267 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:34:14.022288 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:14.022443 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:14.022527 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 02:34:14.022748 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":170,"threads_started":1}
I20260619 02:34:14.024232 7535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cca425719b24cecbe3fd8bf308655e8: Neither blocks nor log segments found. Creating new log.
I20260619 02:34:14.025012 7535 log.cc:824] T 00000000000000000000000000000000 P 3cca425719b24cecbe3fd8bf308655e8: Log is configured to *not* fsync() on all Append() calls
I20260619 02:34:14.026813 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:34:14.027310 7535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cca425719b24cecbe3fd8bf308655e8: No bootstrap required, opened a new log
I20260619 02:34:14.030737 7538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:14.028978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45007 (local address 127.17.96.254:43695)
0619 02:34:14.029119 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:14.029124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:14.029329 (+ 205us) server_negotiation.cc:407] Connection header received
0619 02:34:14.029460 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:14.029462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:14.029494 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:14.029576 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:14.030310 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:14.030313 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:14.030315 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:34:14.030326 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:14.030351 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:34:14.030362 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:14.030433 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:14.030554 (+ 121us) server_negotiation.cc:299] Negotiation successful
0619 02:34:14.030587 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 02:34:14.031984 7535 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cca425719b24cecbe3fd8bf308655e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cca425719b24cecbe3fd8bf308655e8" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 43695 } }
I20260619 02:34:14.032269 7535 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cca425719b24cecbe3fd8bf308655e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:34:14.032330 7535 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cca425719b24cecbe3fd8bf308655e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cca425719b24cecbe3fd8bf308655e8, State: Initialized, Role: FOLLOWER
I20260619 02:34:14.032770 7535 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cca425719b24cecbe3fd8bf308655e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cca425719b24cecbe3fd8bf308655e8" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 43695 } }
I20260619 02:34:14.032922 7535 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cca425719b24cecbe3fd8bf308655e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:34:14.033015 7535 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cca425719b24cecbe3fd8bf308655e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:34:14.033120 7535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cca425719b24cecbe3fd8bf308655e8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:34:14.034224 7535 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cca425719b24cecbe3fd8bf308655e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cca425719b24cecbe3fd8bf308655e8" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 43695 } }
I20260619 02:34:14.034597 7535 leader_election.cc:302] T 00000000000000000000000000000000 P 3cca425719b24cecbe3fd8bf308655e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cca425719b24cecbe3fd8bf308655e8; no voters:
I20260619 02:34:14.034981 7535 leader_election.cc:288] T 00000000000000000000000000000000 P 3cca425719b24cecbe3fd8bf308655e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:34:14.035094 7545 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cca425719b24cecbe3fd8bf308655e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:34:14.035353 7545 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cca425719b24cecbe3fd8bf308655e8 [term 1 LEADER]: Becoming Leader. State: Replica: 3cca425719b24cecbe3fd8bf308655e8, State: Running, Role: LEADER
I20260619 02:34:14.035727 7545 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cca425719b24cecbe3fd8bf308655e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cca425719b24cecbe3fd8bf308655e8" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 43695 } }
I20260619 02:34:14.036517 7535 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cca425719b24cecbe3fd8bf308655e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:34:14.037425 7546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cca425719b24cecbe3fd8bf308655e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cca425719b24cecbe3fd8bf308655e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cca425719b24cecbe3fd8bf308655e8" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 43695 } } }
I20260619 02:34:14.037561 7546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cca425719b24cecbe3fd8bf308655e8 [sys.catalog]: This master's current role is: LEADER
I20260619 02:34:14.038096 7554 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:34:14.038341 7547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cca425719b24cecbe3fd8bf308655e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cca425719b24cecbe3fd8bf308655e8. Latest consensus state: current_term: 1 leader_uuid: "3cca425719b24cecbe3fd8bf308655e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cca425719b24cecbe3fd8bf308655e8" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 43695 } } }
I20260619 02:34:14.038465 7547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cca425719b24cecbe3fd8bf308655e8 [sys.catalog]: This master's current role is: LEADER
I20260619 02:34:14.041702 7554 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:34:14.047703 7554 catalog_manager.cc:1347] Generated new cluster ID: 0c2a50b1966c427a81374b7e389794e0
I20260619 02:34:14.047801 7554 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:34:14.068845 7554 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:34:14.069280 7554 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:34:14.082959 7554 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cca425719b24cecbe3fd8bf308655e8: Generated new TSK 0
I20260619 02:34:14.083987 7554 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:34:14.098328 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 7467
2026-06-19T02:34:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1444 ms)
[----------] 18 tests from AuthTokenIssuingTest (26899 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:34:14 dist-test-slave-4qw6 krb5kdc[7566](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:34:14 dist-test-slave-4qw6 krb5kdc[7566](info): set up 2 sockets
Jun 19 02:34:14 dist-test-slave-4qw6 krb5kdc[7566](info): commencing operation
krb5kdc: starting...
W20260619 02:34:15.158229 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:34:15 dist-test-slave-4qw6 krb5kdc[7566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:34:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:34:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:34:15.288194 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:46021
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:33653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:46021,127.17.96.253:34193,127.17.96.252:37263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:34:15.403100 7579 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:34:15.403438 7579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:34:15.403465 7579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:34:15.403553 7579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:34:15.407922 7579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:34:15.408010 7579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:34:15.408030 7579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:34:15.408054 7579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:34:15.408070 7579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:34:15.413872 7579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:33653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:46021,127.17.96.253:34193,127.17.96.252:37263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:46021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:34:15.415117 7579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:34:15.416201 7579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:34:15.425024 7584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:34:15.425184 7579 server_base.cc:1034] running on GCE node
W20260619 02:34:15.424993 7585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:15.425130 7587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:34:15.425951 7579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:34:15.427188 7579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:34:15.428378 7579 hybrid_clock.cc:648] HybridClock initialized: now 1781836455428347 us; error 47 us; skew 500 ppm
Jun 19 02:34:15 dist-test-slave-4qw6 krb5kdc[7566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836455, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:34:15.431347 7579 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:34:15.432391 7579 webserver.cc:460] Webserver started at http://127.17.96.254:38361/ using document root <none> and password file <none>
I20260619 02:34:15.433012 7579 fs_manager.cc:359] Metadata directory not provided
I20260619 02:34:15.433106 7579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:34:15.433341 7579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:34:15.435606 7579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "998689d6d4b749f7baa7862ccb350c39"
format_stamp: "Formatted at 2026-06-19 02:34:15 on dist-test-slave-4qw6"
server_key: "12229e879cc61007f44e6d5fa9e534b3"
server_key_iv: "0ca3853956b6a442fa0f84e75b8e82a6"
server_key_version: "encryptionkey@0"
I20260619 02:34:15.436218 7579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "998689d6d4b749f7baa7862ccb350c39"
format_stamp: "Formatted at 2026-06-19 02:34:15 on dist-test-slave-4qw6"
server_key: "12229e879cc61007f44e6d5fa9e534b3"
server_key_iv: "0ca3853956b6a442fa0f84e75b8e82a6"
server_key_version: "encryptionkey@0"
I20260619 02:34:15.440330 7579 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260619 02:34:15.443035 7594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:34:15.444474 7579 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:34:15.444655 7579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "998689d6d4b749f7baa7862ccb350c39"
format_stamp: "Formatted at 2026-06-19 02:34:15 on dist-test-slave-4qw6"
server_key: "12229e879cc61007f44e6d5fa9e534b3"
server_key_iv: "0ca3853956b6a442fa0f84e75b8e82a6"
server_key_version: "encryptionkey@0"
I20260619 02:34:15.444792 7579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:34:15.469739 7579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:34:15.473335 7579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:34:15.473567 7579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:34:15.481791 7579 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:46021
I20260619 02:34:15.482161 7646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:46021 every 8 connection(s)
I20260619 02:34:15.482961 7579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:34:15.485438 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 7579
I20260619 02:34:15.485630 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:34:15.486044 17795 external_mini_cluster.cc:1428] Setting key 3808b4adb6ec3a2dde64477583cf1e99
I20260619 02:34:15.487056 7647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:34:15.492280 7647 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46021 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.96.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.253@KRBTEST.COM" created.
Jun 19 02:34:15 dist-test-slave-4qw6 krb5kdc[7566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836455, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:34:15.509887 7650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.494981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:46021 (local address 127.0.0.1:55460)
0619 02:34:15.495565 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.495618 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 02:34:15.496709 (+ 1091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:34:15.497503 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:34:15.497508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:34:15.498096 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:34:15.498437 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:15.498460 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.499671 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.499674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:15.500529 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:15.500541 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.501251 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.501254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:15.501365 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:15.502236 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:34:15.502263 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:34:15.504188 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:34:15.506421 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:15.506440 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:15.506452 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:15.506696 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:15.507046 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:15.507049 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:15.507051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:15.507173 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:15.507849 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:34:15.507857 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:34:15.508005 (+ 148us) client_negotiation.cc:769] Sending connection context
0619 02:34:15.508189 (+ 184us) client_negotiation.cc:240] Negotiation successful
0619 02:34:15.508684 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":135,"thread_start_us":160,"threads_started":1}
I20260619 02:34:15.509889 7651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.494976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55460 (local address 127.17.96.254:46021)
0619 02:34:15.495529 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:15.495551 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:15.495845 (+ 294us) server_negotiation.cc:407] Connection header received
0619 02:34:15.497077 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:15.497082 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:15.497277 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:15.497454 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:15.498626 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.499531 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.500689 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.501082 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.501260 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:15.504384 (+ 3124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:15.504390 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:15.504402 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:15.504425 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:15.506280 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:15.506812 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:15.506819 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:15.506821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:15.506945 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:15.507271 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:15.507274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:15.507276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:15.507718 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:15.507874 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:15.508251 (+ 377us) server_negotiation.cc:299] Negotiation successful
0619 02:34:15.508598 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":399,"spinlock_wait_cycles":25472,"thread_start_us":193,"threads_started":1}
I20260619 02:34:15.512735 7647 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:34:15.513605 7650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.513300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:34193 (local address 127.0.0.1:49244)
0619 02:34:15.513438 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.513533 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.253:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 02:34:15.513994 7598 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.96.253:34193: connect: Connection refused (error 111)
I20260619 02:34:15.514935 7650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.514603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:34193 (local address 127.0.0.1:49246)
0619 02:34:15.514706 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.514752 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.253:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 02:34:15.515115 7647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.253:34193: Network error: Client connection negotiation failed: client connection to 127.17.96.253:34193: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.253.keytab.
Entry for principal kudu/127.17.96.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.253.keytab.
I20260619 02:34:15.516523 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.253:34193
--webserver_interface=127.17.96.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.253.keytab
--principal=kudu/127.17.96.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:33653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:46021,127.17.96.253:34193,127.17.96.252:37263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:34:15.564411 7647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } attempt: 1
I20260619 02:34:15.565372 7650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.565069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:34193 (local address 127.0.0.1:49258)
0619 02:34:15.565229 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.565292 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.253:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260619 02:34:15.566123 7650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.565904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:34193 (local address 127.0.0.1:49264)
0619 02:34:15.566007 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.566064 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.253:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 02:34:15.566249 7647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.253:34193: Network error: Client connection negotiation failed: client connection to 127.17.96.253:34193: connect: Connection refused (error 111)
W20260619 02:34:15.631515 7654 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:34:15.631892 7654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:34:15.631951 7654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:34:15.632046 7654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 02:34:15.634413 7647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } attempt: 2
I20260619 02:34:15.635258 7650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.634952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:34193 (local address 127.0.0.1:49272)
0619 02:34:15.635121 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.635184 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.253:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 02:34:15.635928 7650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.635724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:34193 (local address 127.0.0.1:49280)
0619 02:34:15.635824 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.635875 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.253:34193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 02:34:15.636063 7647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.253:34193: Network error: Client connection negotiation failed: client connection to 127.17.96.253:34193: connect: Connection refused (error 111)
W20260619 02:34:15.636572 7654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:34:15.636653 7654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:34:15.636709 7654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:34:15.636758 7654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:34:15.636795 7654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:34:15.642544 7654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:33653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:46021,127.17.96.253:34193,127.17.96.252:37263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.253.keytab
--principal=kudu/127.17.96.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.253:34193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.96.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:34:15.643739 7654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:34:15.644783 7654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:34:15.653505 7660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:15.653503 7659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:15.653512 7662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:34:15.653868 7654 server_base.cc:1034] running on GCE node
I20260619 02:34:15.654572 7654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:34:15.655726 7654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:34:15.656893 7654 hybrid_clock.cc:648] HybridClock initialized: now 1781836455656881 us; error 29 us; skew 500 ppm
Jun 19 02:34:15 dist-test-slave-4qw6 krb5kdc[7566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836455, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:34:15.659832 7654 init.cc:377] Logged in from keytab as kudu/127.17.96.253@KRBTEST.COM (short username kudu)
I20260619 02:34:15.660833 7654 webserver.cc:460] Webserver started at http://127.17.96.253:43035/ using document root <none> and password file <none>
I20260619 02:34:15.661424 7654 fs_manager.cc:359] Metadata directory not provided
I20260619 02:34:15.661526 7654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:34:15.661811 7654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:34:15.663986 7654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-1/data/instance:
uuid: "1da2735702cb466e941e06432babe553"
format_stamp: "Formatted at 2026-06-19 02:34:15 on dist-test-slave-4qw6"
server_key: "00d5fcbeb17c8eeec3d6fd4a0c4da86f"
server_key_iv: "3b383be90591ac41709b5aec9b6a590d"
server_key_version: "encryptionkey@0"
I20260619 02:34:15.664590 7654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-1/wal/instance:
uuid: "1da2735702cb466e941e06432babe553"
format_stamp: "Formatted at 2026-06-19 02:34:15 on dist-test-slave-4qw6"
server_key: "00d5fcbeb17c8eeec3d6fd4a0c4da86f"
server_key_iv: "3b383be90591ac41709b5aec9b6a590d"
server_key_version: "encryptionkey@0"
I20260619 02:34:15.668674 7654 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260619 02:34:15.671520 7669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:34:15.672866 7654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:34:15.673027 7654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-1/wal
uuid: "1da2735702cb466e941e06432babe553"
format_stamp: "Formatted at 2026-06-19 02:34:15 on dist-test-slave-4qw6"
server_key: "00d5fcbeb17c8eeec3d6fd4a0c4da86f"
server_key_iv: "3b383be90591ac41709b5aec9b6a590d"
server_key_version: "encryptionkey@0"
I20260619 02:34:15.673158 7654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:34:15.694944 7654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:34:15.698098 7654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:34:15.698338 7654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:34:15.705814 7654 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.253:34193
I20260619 02:34:15.706136 7721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.253:34193 every 8 connection(s)
I20260619 02:34:15.706959 7654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-1/data/info.pb
I20260619 02:34:15.710152 7722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:34:15.714037 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 7654
I20260619 02:34:15.714183 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-1/wal/instance
I20260619 02:34:15.714468 17795 external_mini_cluster.cc:1428] Setting key 2affd6949b56a4c4e9fcd76026678245
I20260619 02:34:15.714442 7722 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46021 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.96.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.252@KRBTEST.COM" created.
Jun 19 02:34:15 dist-test-slave-4qw6 krb5kdc[7566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836455, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.253@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:34:15.727236 7647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } attempt: 3
I20260619 02:34:15.731392 7651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.717013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55466 (local address 127.17.96.254:46021)
0619 02:34:15.717168 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:15.717173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:15.718420 (+ 1247us) server_negotiation.cc:407] Connection header received
0619 02:34:15.719564 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:15.719567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:15.719630 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:15.719741 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:15.721071 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.721620 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.722893 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.723180 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.723297 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:15.726824 (+ 3527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:15.726827 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:15.726830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:15.726876 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:15.728789 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:15.729470 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:15.729478 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:15.729480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:15.729579 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:15.730036 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:15.730043 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:15.730048 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:15.730314 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:15.730427 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:15.731070 (+ 643us) server_negotiation.cc:299] Negotiation successful
0619 02:34:15.731214 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:34:15.732635 7725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.717614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:46021 (local address 127.0.0.1:55466)
0619 02:34:15.718252 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.718302 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 02:34:15.719337 (+ 1035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:34:15.719936 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:34:15.719943 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:34:15.720369 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:34:15.720904 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:15.720919 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.721795 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.721803 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:15.722754 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:15.722765 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.723364 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.723368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:15.723564 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:15.724202 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:34:15.724219 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:34:15.726546 (+ 2327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:34:15.728893 (+ 2347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:15.728900 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:15.728919 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:15.729338 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:15.729678 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:15.729680 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:15.729682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:15.729920 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:15.730439 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:34:15.730446 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:34:15.730779 (+ 333us) client_negotiation.cc:769] Sending connection context
0619 02:34:15.730994 (+ 215us) client_negotiation.cc:240] Negotiation successful
0619 02:34:15.731379 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":186,"threads_started":1}
I20260619 02:34:15.734678 7722 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:34:15 dist-test-slave-4qw6 krb5kdc[7566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836455, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for kudu/127.17.96.253@KRBTEST.COM
Jun 19 02:34:15 dist-test-slave-4qw6 krb5kdc[7566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836455, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.253@KRBTEST.COM for kudu/127.17.96.253@KRBTEST.COM
I20260619 02:34:15.740557 7650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.727775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:34193 (local address 127.0.0.1:49284)
0619 02:34:15.728677 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.728696 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:34:15.728789 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:34:15.729326 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:34:15.729331 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:34:15.729530 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:34:15.729687 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:15.729697 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.730942 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.730949 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:15.731872 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:15.731887 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.732712 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.732718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:15.732878 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:15.733442 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:34:15.733463 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:34:15.735950 (+ 2487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:34:15.738450 (+ 2500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:15.738454 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:15.738456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:15.738675 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:15.739114 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:15.739120 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:15.739125 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:15.739238 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:15.739885 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:34:15.739887 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:34:15.739987 (+ 100us) client_negotiation.cc:769] Sending connection context
0619 02:34:15.740118 (+ 131us) client_negotiation.cc:240] Negotiation successful
0619 02:34:15.740333 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777}
I20260619 02:34:15.740662 7727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.728187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49284 (local address 127.17.96.253:34193)
0619 02:34:15.728739 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:15.728761 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:15.728788 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:34:15.728893 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:15.728900 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:15.729158 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:15.729331 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:15.729830 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.730800 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.732027 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.732559 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.732683 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:15.736148 (+ 3465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:15.736163 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:15.736181 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:15.736216 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:15.738321 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:15.738795 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:15.738801 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:15.738803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:15.738983 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:15.739358 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:15.739379 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:15.739383 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:15.739781 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:15.739934 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:15.740249 (+ 315us) server_negotiation.cc:299] Negotiation successful
0619 02:34:15.740435 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"spinlock_wait_cycles":1280,"thread_start_us":114,"threads_started":1}
I20260619 02:34:15.743001 7647 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.252.keytab.
Entry for principal kudu/127.17.96.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.252.keytab.
I20260619 02:34:15.743882 7650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.743601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:37263 (local address 127.0.0.1:53674)
0619 02:34:15.743731 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.743786 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:37263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260619 02:34:15.744789 7650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.744389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:37263 (local address 127.0.0.1:53676)
0619 02:34:15.744651 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.744707 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:37263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191}
W20260619 02:34:15.744939 7647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.252:37263: Network error: Client connection negotiation failed: client connection to 127.17.96.252:37263: connect: Connection refused (error 111)
I20260619 02:34:15.744915 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.252:37263
--webserver_interface=127.17.96.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.252.keytab
--principal=kudu/127.17.96.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:33653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:46021,127.17.96.253:34193,127.17.96.252:37263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:34:15.745445 7725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.735246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:34193 (local address 127.0.0.1:49298)
0619 02:34:15.735610 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.735627 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:34:15.735714 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:34:15.735938 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:34:15.735941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:34:15.736129 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:34:15.736214 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:15.736220 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.737185 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.737190 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:15.737869 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:15.737879 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.738377 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.738381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:15.738474 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:15.738925 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:34:15.738940 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:34:15.740358 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:34:15.743157 (+ 2799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:15.743163 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:15.743184 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:15.743408 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:15.744232 (+ 824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:15.744235 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:15.744238 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:15.744336 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:15.744800 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:34:15.744804 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:34:15.744911 (+ 107us) client_negotiation.cc:769] Sending connection context
0619 02:34:15.745033 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 02:34:15.745196 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289}
I20260619 02:34:15.746104 7728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.735264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49298 (local address 127.17.96.253:34193)
0619 02:34:15.735661 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:15.735672 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:15.735685 (+ 13us) server_negotiation.cc:407] Connection header received
0619 02:34:15.735800 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:15.735802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:15.735843 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:15.735930 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:15.736322 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.737066 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.737978 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.738269 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.738378 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:15.740512 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:15.740518 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:15.740523 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:15.740546 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:15.743016 (+ 2470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:15.743511 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:15.743514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:15.743515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:15.743613 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:15.744435 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:15.744441 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:15.744444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:15.744689 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:15.744793 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:15.745669 (+ 876us) server_negotiation.cc:299] Negotiation successful
0619 02:34:15.745842 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":204,"threads_started":1}
I20260619 02:34:15.746748 7722 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:34:15.747475 7725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.747215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:37263 (local address 127.0.0.1:53692)
0619 02:34:15.747330 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.747418 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:37263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260619 02:34:15.747707 7671 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.96.252:37263: connect: Connection refused (error 111)
I20260619 02:34:15.748658 7725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.748436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:37263 (local address 127.0.0.1:53704)
0619 02:34:15.748542 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.748607 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:37263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 02:34:15.748813 7722 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.252:37263: Network error: Client connection negotiation failed: client connection to 127.17.96.252:37263: connect: Connection refused (error 111)
I20260619 02:34:15.776103 7647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } attempt: 1
I20260619 02:34:15.777091 7650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.776726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:37263 (local address 127.0.0.1:53720)
0619 02:34:15.776940 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.777007 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:37263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260619 02:34:15.777926 7650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.777629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:37263 (local address 127.0.0.1:53724)
0619 02:34:15.777735 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.777852 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:37263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 02:34:15.778097 7647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.252:37263: Network error: Client connection negotiation failed: client connection to 127.17.96.252:37263: connect: Connection refused (error 111)
I20260619 02:34:15.798056 7722 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } attempt: 1
I20260619 02:34:15.798990 7725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.798669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:37263 (local address 127.0.0.1:53728)
0619 02:34:15.798824 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.798895 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:37263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260619 02:34:15.799675 7725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.799478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:37263 (local address 127.0.0.1:53736)
0619 02:34:15.799580 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.799627 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:37263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 02:34:15.799798 7722 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.252:37263: Network error: Client connection negotiation failed: client connection to 127.17.96.252:37263: connect: Connection refused (error 111)
I20260619 02:34:15.853271 7647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } attempt: 2
I20260619 02:34:15.854171 7650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.853881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:37263 (local address 127.0.0.1:53746)
0619 02:34:15.854028 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.854092 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:37263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260619 02:34:15.854902 7650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.854697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:37263 (local address 127.0.0.1:53748)
0619 02:34:15.854797 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.854847 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:37263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 02:34:15.855031 7647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.252:37263: Network error: Client connection negotiation failed: client connection to 127.17.96.252:37263: connect: Connection refused (error 111)
W20260619 02:34:15.862571 7729 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:34:15.862931 7729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:34:15.862989 7729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:34:15.863081 7729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:34:15.867326 7729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:34:15.867421 7729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:34:15.867458 7729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:34:15.867494 7729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:34:15.867529 7729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:34:15.867967 7722 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } attempt: 2
I20260619 02:34:15.868789 7725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.868493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:37263 (local address 127.0.0.1:53754)
0619 02:34:15.868644 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.868713 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:37263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260619 02:34:15.869470 7725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.869264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:37263 (local address 127.0.0.1:53768)
0619 02:34:15.869372 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.869420 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:37263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 02:34:15.869588 7722 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.252:37263: Network error: Client connection negotiation failed: client connection to 127.17.96.252:37263: connect: Connection refused (error 111)
I20260619 02:34:15.873450 7729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:33653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:46021,127.17.96.253:34193,127.17.96.252:37263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.252.keytab
--principal=kudu/127.17.96.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.252:37263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.96.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:34:15.874670 7729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:34:15.875703 7729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:34:15.884615 7735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:15.884615 7736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:15.884635 7738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:34:15.884884 7729 server_base.cc:1034] running on GCE node
I20260619 02:34:15.885682 7729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:34:15.886925 7729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:34:15.888087 7729 hybrid_clock.cc:648] HybridClock initialized: now 1781836455888070 us; error 33 us; skew 500 ppm
Jun 19 02:34:15 dist-test-slave-4qw6 krb5kdc[7566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836455, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:34:15.890908 7729 init.cc:377] Logged in from keytab as kudu/127.17.96.252@KRBTEST.COM (short username kudu)
I20260619 02:34:15.891958 7729 webserver.cc:460] Webserver started at http://127.17.96.252:33789/ using document root <none> and password file <none>
I20260619 02:34:15.892566 7729 fs_manager.cc:359] Metadata directory not provided
I20260619 02:34:15.892658 7729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:34:15.892886 7729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:34:15.895140 7729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-2/data/instance:
uuid: "3fc78b8b166c44e99b2c58bbf49dda4a"
format_stamp: "Formatted at 2026-06-19 02:34:15 on dist-test-slave-4qw6"
server_key: "d3aff1d8061ea79882967c3901f7de2c"
server_key_iv: "65702f2cb9c22f5dcb48b3c8669463df"
server_key_version: "encryptionkey@0"
I20260619 02:34:15.895752 7729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-2/wal/instance:
uuid: "3fc78b8b166c44e99b2c58bbf49dda4a"
format_stamp: "Formatted at 2026-06-19 02:34:15 on dist-test-slave-4qw6"
server_key: "d3aff1d8061ea79882967c3901f7de2c"
server_key_iv: "65702f2cb9c22f5dcb48b3c8669463df"
server_key_version: "encryptionkey@0"
I20260619 02:34:15.899787 7729 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:34:15.902354 7745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:34:15.903627 7729 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 02:34:15.903784 7729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-2/wal
uuid: "3fc78b8b166c44e99b2c58bbf49dda4a"
format_stamp: "Formatted at 2026-06-19 02:34:15 on dist-test-slave-4qw6"
server_key: "d3aff1d8061ea79882967c3901f7de2c"
server_key_iv: "65702f2cb9c22f5dcb48b3c8669463df"
server_key_version: "encryptionkey@0"
I20260619 02:34:15.903909 7729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:34:15.930148 7729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:34:15.933574 7729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:34:15.933851 7729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:34:15.941663 7729 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.252:37263
I20260619 02:34:15.941983 7797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.252:37263 every 8 connection(s)
I20260619 02:34:15.942828 7729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-2/data/info.pb
I20260619 02:34:15.945729 7798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:34:15.950158 7798 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46021 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:34:15.952584 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 7729
I20260619 02:34:15.952744 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781836162001484-17795-0/minicluster-data/master-2/wal/instance
I20260619 02:34:15.953001 17795 external_mini_cluster.cc:1428] Setting key f985dbf22c348db2a8bc56132bddf406
I20260619 02:34:15.954195 7647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } attempt: 3
Jun 19 02:34:15 dist-test-slave-4qw6 krb5kdc[7566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836455, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for kudu/127.17.96.252@KRBTEST.COM
I20260619 02:34:15.960757 7722 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } attempt: 3
Jun 19 02:34:15 dist-test-slave-4qw6 krb5kdc[7566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836455, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.252@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
Jun 19 02:34:15 dist-test-slave-4qw6 krb5kdc[7566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.252@KRBTEST.COM
I20260619 02:34:15.966410 7650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.954649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:37263 (local address 127.0.0.1:53790)
0619 02:34:15.954752 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.954769 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:34:15.954868 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:34:15.955263 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:34:15.955266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:34:15.955498 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:34:15.955631 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:15.955642 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.957238 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.957242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:15.958100 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:15.958111 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.958806 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.958813 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:15.958902 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:15.959496 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:34:15.959514 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:34:15.961030 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:34:15.963876 (+ 2846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:15.963881 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:15.963884 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:15.964546 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:15.964930 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:15.964934 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:15.964936 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:15.965029 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:15.965854 (+ 825us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:34:15.965857 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:34:15.965953 (+ 96us) client_negotiation.cc:769] Sending connection context
0619 02:34:15.966041 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 02:34:15.966208 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 02:34:15.966934 7651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.952216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55480 (local address 127.17.96.254:46021)
0619 02:34:15.952376 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:15.952381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:15.953507 (+ 1126us) server_negotiation.cc:407] Connection header received
0619 02:34:15.955087 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:15.955094 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:15.955151 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:15.955218 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:15.956315 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.957072 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.958290 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.958622 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.958740 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:15.962526 (+ 3786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:15.962532 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:15.962536 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:15.962564 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:15.964253 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:15.964889 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:15.964893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:15.964897 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:15.964987 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:15.965373 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:15.965378 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:15.965380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:15.965708 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:15.965841 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:15.966674 (+ 833us) server_negotiation.cc:299] Negotiation successful
0619 02:34:15.966800 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 02:34:15.968247 7800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.952627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:46021 (local address 127.0.0.1:55480)
0619 02:34:15.953323 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.953392 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0619 02:34:15.954572 (+ 1180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:34:15.955386 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:34:15.955391 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:34:15.955805 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:34:15.956167 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:15.956181 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.957212 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.957217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:15.958160 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:15.958188 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.958792 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.958796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:15.959345 (+ 549us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:15.959890 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:34:15.959912 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:34:15.962338 (+ 2426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:34:15.964382 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:15.964394 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:15.964409 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:15.964759 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:15.965100 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:15.965103 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:15.965105 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:15.965255 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:15.965910 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:34:15.965920 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:34:15.966094 (+ 174us) client_negotiation.cc:769] Sending connection context
0619 02:34:15.967117 (+ 1023us) client_negotiation.cc:240] Negotiation successful
0619 02:34:15.967300 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":451,"thread_start_us":254,"threads_started":1}
I20260619 02:34:15.968264 7803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.954711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53790 (local address 127.17.96.252:37263)
0619 02:34:15.955021 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:15.955032 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:15.955044 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:34:15.955074 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:15.955077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:15.955122 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:15.955760 (+ 638us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:15.955958 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.957127 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.958225 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.958648 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.959236 (+ 588us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:15.961176 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:15.961183 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:15.961198 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:15.961478 (+ 280us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:15.963737 (+ 2259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:15.964665 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:15.964671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:15.964673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:15.964821 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:15.965134 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:15.965137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:15.965139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:15.965686 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:15.966628 (+ 942us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:15.966759 (+ 131us) server_negotiation.cc:299] Negotiation successful
0619 02:34:15.967381 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":172,"server-negotiator.queue_time_us":230,"thread_start_us":101,"threads_started":1}
I20260619 02:34:15.968247 7801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.953691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53774 (local address 127.17.96.252:37263)
0619 02:34:15.954073 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:15.954083 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:15.954395 (+ 312us) server_negotiation.cc:407] Connection header received
0619 02:34:15.954723 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:15.954745 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:15.954997 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:15.955216 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:15.956076 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.957026 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.958279 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.958670 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.958849 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:15.962667 (+ 3818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:15.962676 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:15.962679 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:15.962710 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:15.965045 (+ 2335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:15.965567 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:15.965570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:15.965571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:15.965651 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:15.965998 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:15.966001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:15.966002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:15.966204 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:15.966454 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:15.966763 (+ 309us) server_negotiation.cc:299] Negotiation successful
0619 02:34:15.967102 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":269,"thread_start_us":174,"threads_started":1}
Jun 19 02:34:15 dist-test-slave-4qw6 krb5kdc[7566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836455, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.253@KRBTEST.COM for kudu/127.17.96.252@KRBTEST.COM
I20260619 02:34:15.970679 7798 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:34:15.973582 7725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.961247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:37263 (local address 127.0.0.1:53792)
0619 02:34:15.961411 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.961430 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:34:15.961522 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:34:15.962409 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:34:15.962413 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:34:15.962588 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:34:15.962715 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:15.962724 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.964438 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.964442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:15.965260 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:15.965271 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.965871 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.965875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:15.965972 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:15.967239 (+ 1267us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:34:15.967255 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:34:15.969167 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:34:15.971845 (+ 2678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:15.971849 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:15.971852 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:15.972155 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:15.972489 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:15.972492 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:15.972494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:15.972592 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:15.973019 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:34:15.973022 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:34:15.973128 (+ 106us) client_negotiation.cc:769] Sending connection context
0619 02:34:15.973237 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 02:34:15.973381 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260619 02:34:15.973582 7647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39: Bootstrap starting.
I20260619 02:34:15.973647 7804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.961284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53792 (local address 127.17.96.252:37263)
0619 02:34:15.961569 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:15.961580 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:15.961593 (+ 13us) server_negotiation.cc:407] Connection header received
0619 02:34:15.961632 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:15.961634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:15.961681 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:15.962831 (+ 1150us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:15.962942 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.964310 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.965395 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.965702 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.966191 (+ 489us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:15.969312 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:15.969316 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:15.969319 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:15.969346 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:15.971707 (+ 2361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:15.972266 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:15.972270 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:15.972272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:15.972365 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:15.972692 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:15.972695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:15.972697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:15.972920 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:15.973016 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:15.973305 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 02:34:15.973438 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":115,"threads_started":1}
I20260619 02:34:15.976466 7647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39: Neither blocks nor log segments found. Creating new log.
Jun 19 02:34:15 dist-test-slave-4qw6 krb5kdc[7566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:34:15.977239 7722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1da2735702cb466e941e06432babe553: Bootstrap starting.
I20260619 02:34:15.977308 7647 log.cc:824] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39: Log is configured to *not* fsync() on all Append() calls
I20260619 02:34:15.979984 7647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39: No bootstrap required, opened a new log
I20260619 02:34:15.980329 7722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1da2735702cb466e941e06432babe553: Neither blocks nor log segments found. Creating new log.
I20260619 02:34:15.981252 7722 log.cc:824] T 00000000000000000000000000000000 P 1da2735702cb466e941e06432babe553: Log is configured to *not* fsync() on all Append() calls
I20260619 02:34:15.981998 7651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.971072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55484 (local address 127.17.96.254:46021)
0619 02:34:15.971230 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:15.971234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:15.971250 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:34:15.971281 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:15.971283 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:15.971318 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:15.971409 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:15.972145 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.972841 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.974421 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.974735 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.974853 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:15.978002 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:15.978006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:15.978009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:15.978039 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:15.979795 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:15.980339 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:15.980346 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:15.980350 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:15.980476 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:15.980813 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:15.980816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:15.980818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:15.981045 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:15.981209 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:15.981619 (+ 410us) server_negotiation.cc:299] Negotiation successful
0619 02:34:15.981802 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260619 02:34:15.983608 7722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1da2735702cb466e941e06432babe553: No bootstrap required, opened a new log
Jun 19 02:34:15 dist-test-slave-4qw6 krb5kdc[7566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836455, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.252@KRBTEST.COM for kudu/127.17.96.253@KRBTEST.COM
I20260619 02:34:15.984241 7647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998689d6d4b749f7baa7862ccb350c39" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46021 } } peers { permanent_uuid: "1da2735702cb466e941e06432babe553" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } } peers { permanent_uuid: "3fc78b8b166c44e99b2c58bbf49dda4a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } }
I20260619 02:34:15.984526 7647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998689d6d4b749f7baa7862ccb350c39, State: Initialized, Role: FOLLOWER
I20260619 02:34:15.984926 7647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998689d6d4b749f7baa7862ccb350c39" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46021 } } peers { permanent_uuid: "1da2735702cb466e941e06432babe553" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } } peers { permanent_uuid: "3fc78b8b166c44e99b2c58bbf49dda4a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } }
I20260619 02:34:15.986349 7807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998689d6d4b749f7baa7862ccb350c39" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46021 } } peers { permanent_uuid: "1da2735702cb466e941e06432babe553" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } } peers { permanent_uuid: "3fc78b8b166c44e99b2c58bbf49dda4a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } } }
I20260619 02:34:15.986526 7807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:34:15.986608 7647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:34:15.987957 7722 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1da2735702cb466e941e06432babe553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998689d6d4b749f7baa7862ccb350c39" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46021 } } peers { permanent_uuid: "1da2735702cb466e941e06432babe553" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } } peers { permanent_uuid: "3fc78b8b166c44e99b2c58bbf49dda4a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } }
I20260619 02:34:15.988222 7800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.977994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:34193 (local address 127.0.0.1:49302)
0619 02:34:15.978316 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.978333 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:34:15.978423 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:34:15.978769 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:34:15.978772 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:34:15.978953 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:34:15.979060 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:15.979068 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.980170 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.980174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:15.980971 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:15.980983 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.981922 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.981926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:15.982028 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:15.982518 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:34:15.982533 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:34:15.984085 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:34:15.986215 (+ 2130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:15.986219 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:15.986221 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:15.986619 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:15.986981 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:15.986984 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:15.986986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:15.987087 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:15.987658 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:34:15.987662 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:34:15.987772 (+ 110us) client_negotiation.cc:769] Sending connection context
0619 02:34:15.987889 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 02:34:15.988054 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238}
I20260619 02:34:15.988221 7722 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1da2735702cb466e941e06432babe553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1da2735702cb466e941e06432babe553, State: Initialized, Role: FOLLOWER
I20260619 02:34:15.988255 7728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.978282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49302 (local address 127.17.96.253:34193)
0619 02:34:15.978396 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:15.978400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:15.978414 (+ 14us) server_negotiation.cc:407] Connection header received
0619 02:34:15.978516 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:15.978519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:15.978571 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:15.978647 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:15.979197 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.979952 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.981213 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.981564 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.981667 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:15.984263 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:15.984269 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:15.984273 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:15.984303 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:15.986065 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:15.986772 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:15.986776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:15.986778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:15.986871 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:15.987200 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:15.987203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:15.987205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:15.987547 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:15.987669 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:15.987947 (+ 278us) server_negotiation.cc:299] Negotiation successful
0619 02:34:15.988097 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 02:34:15.988631 7722 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1da2735702cb466e941e06432babe553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998689d6d4b749f7baa7862ccb350c39" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46021 } } peers { permanent_uuid: "1da2735702cb466e941e06432babe553" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } } peers { permanent_uuid: "3fc78b8b166c44e99b2c58bbf49dda4a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } }
I20260619 02:34:15.989006 7798 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:34:15.989178 7818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1da2735702cb466e941e06432babe553 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998689d6d4b749f7baa7862ccb350c39" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46021 } } peers { permanent_uuid: "1da2735702cb466e941e06432babe553" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } } peers { permanent_uuid: "3fc78b8b166c44e99b2c58bbf49dda4a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } } }
I20260619 02:34:15.989351 7818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1da2735702cb466e941e06432babe553 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 02:34:15.993024 7819 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:34:15.993144 7819 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Jun 19 02:34:15 dist-test-slave-4qw6 krb5kdc[7566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836455, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.252@KRBTEST.COM for kudu/127.17.96.252@KRBTEST.COM
I20260619 02:34:15.994875 7579 master_runner.cc:186] Error getting master registration for 127.17.96.253:34193: OK, instance_id { permanent_uuid: "1da2735702cb466e941e06432babe553" instance_seqno: 1781836455703611 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:34:15.999646 7579 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Jun 19 02:34:15 dist-test-slave-4qw6 krb5kdc[7566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.253@KRBTEST.COM
I20260619 02:34:15.999730 7800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.989553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:37263 (local address 127.0.0.1:53806)
0619 02:34:15.989695 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:15.989714 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:34:15.989844 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:34:15.990172 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:34:15.990176 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:34:15.990368 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:34:15.990508 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:15.990518 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.991565 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.991568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:15.992342 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:15.992356 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.993055 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.993061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:15.993152 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:15.993742 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:34:15.993792 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:34:15.995535 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:34:15.997859 (+ 2324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:15.997863 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:15.997865 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:15.998146 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:15.998481 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:15.998485 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:15.998487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:15.998586 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:15.999126 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:34:15.999128 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:34:15.999232 (+ 104us) client_negotiation.cc:769] Sending connection context
0619 02:34:15.999339 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 02:34:15.999528 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260619 02:34:16.000447 7804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.989815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53806 (local address 127.17.96.252:37263)
0619 02:34:15.989915 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:15.989919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:15.989932 (+ 13us) server_negotiation.cc:407] Connection header received
0619 02:34:15.989968 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:15.989971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:15.990034 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:15.990119 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:15.990650 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.991435 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.992487 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.992812 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.992911 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:15.995683 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:15.995687 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:15.995689 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:15.995715 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:15.997668 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:15.998277 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:15.998282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:15.998285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:15.998375 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:15.998695 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:15.998700 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:15.998702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:15.998941 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:15.999070 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:16.000132 (+ 1062us) server_negotiation.cc:299] Negotiation successful
0619 02:34:16.000270 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:34:16.000427 7722 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1da2735702cb466e941e06432babe553 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:34:16.004436 7798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fc78b8b166c44e99b2c58bbf49dda4a: Bootstrap starting.
W20260619 02:34:16.006635 7831 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1da2735702cb466e941e06432babe553: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:34:16.006748 7831 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:34:16.008669 7798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fc78b8b166c44e99b2c58bbf49dda4a: Neither blocks nor log segments found. Creating new log.
I20260619 02:34:16.009694 7798 log.cc:824] T 00000000000000000000000000000000 P 3fc78b8b166c44e99b2c58bbf49dda4a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:34:16.010382 7728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:15.993027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49316 (local address 127.17.96.253:34193)
0619 02:34:15.993157 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:15.993163 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:15.993181 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:34:15.993229 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:15.993232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:15.993283 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:15.993375 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:15.994327 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.995206 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.996399 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:15.996807 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:15.996922 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:16.000450 (+ 3528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:16.000458 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:16.000465 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:16.000501 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:16.007522 (+ 7021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:16.008249 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:16.008253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:16.008255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:16.008413 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:16.008845 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:16.008849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:16.008850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:16.009131 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:16.009348 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:16.010111 (+ 763us) server_negotiation.cc:299] Negotiation successful
0619 02:34:16.010252 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 02:34:16.009516 7654 master_runner.cc:186] Error getting master registration for 127.17.96.252:37263: OK, instance_id { permanent_uuid: "3fc78b8b166c44e99b2c58bbf49dda4a" instance_seqno: 1781836455939257 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:34:16.013006 7654 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:34:16.016577 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:34:16.017903 7621 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "998689d6d4b749f7baa7862ccb350c39"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55484
I20260619 02:34:16.018189 7621 raft_consensus.cc:482] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 02:34:16.018309 7621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:34:16.019737 7621 raft_consensus.cc:504] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998689d6d4b749f7baa7862ccb350c39" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46021 } } peers { permanent_uuid: "1da2735702cb466e941e06432babe553" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } } peers { permanent_uuid: "3fc78b8b166c44e99b2c58bbf49dda4a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } }
I20260619 02:34:16.020187 7798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fc78b8b166c44e99b2c58bbf49dda4a: No bootstrap required, opened a new log
I20260619 02:34:16.021173 7621 leader_election.cc:288] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39 [CANDIDATE]: Term 1 election: Requested vote from peers 1da2735702cb466e941e06432babe553 (127.17.96.253:34193), 3fc78b8b166c44e99b2c58bbf49dda4a (127.17.96.252:37263)
I20260619 02:34:16.021595 7696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "998689d6d4b749f7baa7862ccb350c39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1da2735702cb466e941e06432babe553"
I20260619 02:34:16.021901 7696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1da2735702cb466e941e06432babe553 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:34:16.023355 7696 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1da2735702cb466e941e06432babe553 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998689d6d4b749f7baa7862ccb350c39 in term 1.
I20260619 02:34:16.023831 7598 leader_election.cc:302] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1da2735702cb466e941e06432babe553, 998689d6d4b749f7baa7862ccb350c39; no voters:
I20260619 02:34:16.025650 7807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:34:16.025919 7807 raft_consensus.cc:686] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39 [term 1 LEADER]: Becoming Leader. State: Replica: 998689d6d4b749f7baa7862ccb350c39, State: Running, Role: LEADER
I20260619 02:34:16.026341 7807 consensus_queue.cc:237] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998689d6d4b749f7baa7862ccb350c39" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46021 } } peers { permanent_uuid: "1da2735702cb466e941e06432babe553" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } } peers { permanent_uuid: "3fc78b8b166c44e99b2c58bbf49dda4a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } }
I20260619 02:34:16.028276 7807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 998689d6d4b749f7baa7862ccb350c39. Latest consensus state: current_term: 1 leader_uuid: "998689d6d4b749f7baa7862ccb350c39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998689d6d4b749f7baa7862ccb350c39" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46021 } } peers { permanent_uuid: "1da2735702cb466e941e06432babe553" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } } peers { permanent_uuid: "3fc78b8b166c44e99b2c58bbf49dda4a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } } }
I20260619 02:34:16.028389 7807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39 [sys.catalog]: This master's current role is: LEADER
I20260619 02:34:16.034101 7842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:34:16.035200 7842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:34:16.033620 7798 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fc78b8b166c44e99b2c58bbf49dda4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998689d6d4b749f7baa7862ccb350c39" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46021 } } peers { permanent_uuid: "1da2735702cb466e941e06432babe553" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } } peers { permanent_uuid: "3fc78b8b166c44e99b2c58bbf49dda4a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } }
I20260619 02:34:16.039093 7798 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fc78b8b166c44e99b2c58bbf49dda4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc78b8b166c44e99b2c58bbf49dda4a, State: Initialized, Role: FOLLOWER
I20260619 02:34:16.039536 7798 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fc78b8b166c44e99b2c58bbf49dda4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998689d6d4b749f7baa7862ccb350c39" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46021 } } peers { permanent_uuid: "1da2735702cb466e941e06432babe553" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } } peers { permanent_uuid: "3fc78b8b166c44e99b2c58bbf49dda4a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } }
I20260619 02:34:16.041163 7798 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fc78b8b166c44e99b2c58bbf49dda4a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:34:16.041249 7772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "998689d6d4b749f7baa7862ccb350c39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3fc78b8b166c44e99b2c58bbf49dda4a"
I20260619 02:34:16.041501 7772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fc78b8b166c44e99b2c58bbf49dda4a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:34:16.043321 7772 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3fc78b8b166c44e99b2c58bbf49dda4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 998689d6d4b749f7baa7862ccb350c39 in term 1.
I20260619 02:34:16.046599 7696 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1da2735702cb466e941e06432babe553 [term 1 FOLLOWER]: Refusing update from remote peer 998689d6d4b749f7baa7862ccb350c39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:34:16.052781 7772 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3fc78b8b166c44e99b2c58bbf49dda4a [term 1 FOLLOWER]: Refusing update from remote peer 998689d6d4b749f7baa7862ccb350c39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:34:16.053522 7838 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1da2735702cb466e941e06432babe553" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:34:16.053844 7807 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fc78b8b166c44e99b2c58bbf49dda4a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:34:16.060905 7845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fc78b8b166c44e99b2c58bbf49dda4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "998689d6d4b749f7baa7862ccb350c39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998689d6d4b749f7baa7862ccb350c39" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46021 } } peers { permanent_uuid: "1da2735702cb466e941e06432babe553" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } } peers { permanent_uuid: "3fc78b8b166c44e99b2c58bbf49dda4a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } } }
I20260619 02:34:16.061091 7845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fc78b8b166c44e99b2c58bbf49dda4a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:34:16.061203 7845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fc78b8b166c44e99b2c58bbf49dda4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 998689d6d4b749f7baa7862ccb350c39. Latest consensus state: current_term: 1 leader_uuid: "998689d6d4b749f7baa7862ccb350c39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998689d6d4b749f7baa7862ccb350c39" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46021 } } peers { permanent_uuid: "1da2735702cb466e941e06432babe553" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } } peers { permanent_uuid: "3fc78b8b166c44e99b2c58bbf49dda4a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } } }
I20260619 02:34:16.061331 7845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fc78b8b166c44e99b2c58bbf49dda4a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:34:16.063766 7818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1da2735702cb466e941e06432babe553 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 998689d6d4b749f7baa7862ccb350c39. Latest consensus state: current_term: 1 leader_uuid: "998689d6d4b749f7baa7862ccb350c39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998689d6d4b749f7baa7862ccb350c39" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46021 } } peers { permanent_uuid: "1da2735702cb466e941e06432babe553" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } } peers { permanent_uuid: "3fc78b8b166c44e99b2c58bbf49dda4a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } } }
I20260619 02:34:16.063915 7818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1da2735702cb466e941e06432babe553 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 02:34:16.064715 7856 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3fc78b8b166c44e99b2c58bbf49dda4a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:34:16.064824 7856 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:34:16.072340 7651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:16.025786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55488 (local address 127.17.96.254:46021)
0619 02:34:16.025923 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:16.025929 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:16.044503 (+ 18574us) server_negotiation.cc:407] Connection header received
0619 02:34:16.044764 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:16.044767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:16.044826 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:16.044888 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:16.046783 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:16.047514 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:16.048760 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:16.049098 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:16.049177 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:16.066709 (+ 17532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:16.066715 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:16.066719 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:16.066756 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:16.068834 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:16.069630 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:16.069637 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:16.069639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:16.069932 (+ 293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:16.070767 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:16.070772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:16.070774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:16.071098 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:16.071251 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:16.071949 (+ 698us) server_negotiation.cc:299] Negotiation successful
0619 02:34:16.072119 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 02:34:16.074733 7728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:16.026575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49332 (local address 127.17.96.253:34193)
0619 02:34:16.026702 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:16.026706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:16.050294 (+ 23588us) server_negotiation.cc:407] Connection header received
0619 02:34:16.050475 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:16.050478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:16.050527 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:16.050599 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:16.051662 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:16.052345 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:16.054788 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:16.055100 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:16.055183 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:16.069404 (+ 14221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:16.069410 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:16.069414 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:16.069456 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:16.072452 (+ 2996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:16.073100 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:16.073105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:16.073107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:16.073219 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:16.073569 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:16.073573 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:16.073574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:16.073846 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:16.073975 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:16.074364 (+ 389us) server_negotiation.cc:299] Negotiation successful
0619 02:34:16.074511 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 02:34:16.076615 7804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:16.038139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53820 (local address 127.17.96.252:37263)
0619 02:34:16.038280 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:16.038286 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:16.056189 (+ 17903us) server_negotiation.cc:407] Connection header received
0619 02:34:16.056366 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:16.056369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:16.056420 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:16.056469 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:16.057502 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:16.058250 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:16.059345 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:16.059668 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:16.059750 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:16.068881 (+ 9131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:16.068889 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:16.068894 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:16.068932 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:16.071091 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:16.074571 (+ 3480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:16.074575 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:16.074577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:16.074679 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:16.075429 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:16.075434 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:16.075454 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:16.075713 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:16.075852 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:16.076280 (+ 428us) server_negotiation.cc:299] Negotiation successful
0619 02:34:16.076425 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 02:34:16.079564 7807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "998689d6d4b749f7baa7862ccb350c39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998689d6d4b749f7baa7862ccb350c39" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46021 } } peers { permanent_uuid: "1da2735702cb466e941e06432babe553" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } } peers { permanent_uuid: "3fc78b8b166c44e99b2c58bbf49dda4a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } } }
I20260619 02:34:16.079725 7807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39 [sys.catalog]: This master's current role is: LEADER
I20260619 02:34:16.080214 7843 mvcc.cc:205] Tried to move back new op lower bound from 7298402123944484864 to 7298402123888046080. Current Snapshot: MvccSnapshot[applied={T|T < 7298402123944484864}]
I20260619 02:34:16.080248 7857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fc78b8b166c44e99b2c58bbf49dda4a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "998689d6d4b749f7baa7862ccb350c39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998689d6d4b749f7baa7862ccb350c39" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46021 } } peers { permanent_uuid: "1da2735702cb466e941e06432babe553" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } } peers { permanent_uuid: "3fc78b8b166c44e99b2c58bbf49dda4a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } } }
I20260619 02:34:16.080404 7857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fc78b8b166c44e99b2c58bbf49dda4a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:34:16.080726 7858 mvcc.cc:205] Tried to move back new op lower bound from 7298402123944484864 to 7298402123888046080. Current Snapshot: MvccSnapshot[applied={T|T < 7298402123944484864}]
I20260619 02:34:16.089380 7838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "998689d6d4b749f7baa7862ccb350c39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998689d6d4b749f7baa7862ccb350c39" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46021 } } peers { permanent_uuid: "1da2735702cb466e941e06432babe553" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } } peers { permanent_uuid: "3fc78b8b166c44e99b2c58bbf49dda4a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } } }
I20260619 02:34:16.089567 7838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39 [sys.catalog]: This master's current role is: LEADER
I20260619 02:34:16.091140 7818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1da2735702cb466e941e06432babe553 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "998689d6d4b749f7baa7862ccb350c39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "998689d6d4b749f7baa7862ccb350c39" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 46021 } } peers { permanent_uuid: "1da2735702cb466e941e06432babe553" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 34193 } } peers { permanent_uuid: "3fc78b8b166c44e99b2c58bbf49dda4a" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 37263 } } }
I20260619 02:34:16.091286 7818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1da2735702cb466e941e06432babe553 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:34:16.093793 7842 catalog_manager.cc:1347] Generated new cluster ID: 7c1f0a5ae99d45b5b978792e845a4710
I20260619 02:34:16.093874 7842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:34:16.095340 7860 mvcc.cc:205] Tried to move back new op lower bound from 7298402123944484864 to 7298402123888046080. Current Snapshot: MvccSnapshot[applied={T|T < 7298402123944484864}]
I20260619 02:34:16.114326 7842 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:34:16.115805 7842 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:34:16.146508 7842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 998689d6d4b749f7baa7862ccb350c39: Generated new TSK 0
I20260619 02:34:16.147485 7842 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:34:16.177453 7651 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:16.173698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55494 (local address 127.17.96.254:46021)
0619 02:34:16.173855 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:16.173861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:16.175842 (+ 1981us) server_negotiation.cc:407] Connection header received
0619 02:34:16.176309 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:16.176313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:16.176400 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:16.176474 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:16.177279 (+ 805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55494: BlockingRecv error: recv got EOF from 127.0.0.1:55494 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260619 02:34:16.178865 7728 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:16.174249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49348 (local address 127.17.96.253:34193)
0619 02:34:16.174391 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:16.174396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:16.176211 (+ 1815us) server_negotiation.cc:407] Connection header received
0619 02:34:16.177933 (+ 1722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:16.177938 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:16.178002 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:16.178064 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:16.178726 (+ 662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49348: BlockingRecv error: recv got EOF from 127.0.0.1:49348 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260619 02:34:16.180495 7804 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:16.175220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53836 (local address 127.17.96.252:37263)
0619 02:34:16.175359 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:16.175364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:16.179479 (+ 4115us) server_negotiation.cc:407] Connection header received
0619 02:34:16.179602 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:16.179605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:16.179682 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:16.179773 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:16.180344 (+ 571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53836: BlockingRecv error: recv got EOF from 127.0.0.1:53836 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260619 02:34:16.185657 7728 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:16.183450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49364 (local address 127.17.96.253:34193)
0619 02:34:16.183604 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:16.183609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:16.184296 (+ 687us) server_negotiation.cc:407] Connection header received
0619 02:34:16.184600 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:16.184604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:16.184700 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:16.184770 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:16.185517 (+ 747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49364: BlockingRecv error: recv got EOF from 127.0.0.1:49364 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260619 02:34:16.192566 7728 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:16.190501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49370 (local address 127.17.96.253:34193)
0619 02:34:16.190647 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:16.190652 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:16.191355 (+ 703us) server_negotiation.cc:407] Connection header received
0619 02:34:16.191633 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:16.191637 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:16.191704 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:16.191767 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:16.192435 (+ 668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49370: BlockingRecv error: recv got EOF from 127.0.0.1:49370 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260619 02:34:16.199183 7728 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:16.197149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49380 (local address 127.17.96.253:34193)
0619 02:34:16.197285 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:16.197290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:16.198051 (+ 761us) server_negotiation.cc:407] Connection header received
0619 02:34:16.198268 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:16.198272 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:16.198331 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:16.198404 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:16.199047 (+ 643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49380: BlockingRecv error: recv got EOF from 127.0.0.1:49380 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260619 02:34:16.207660 7728 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:16.205786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49390 (local address 127.17.96.253:34193)
0619 02:34:16.205965 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:16.205970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:16.206578 (+ 608us) server_negotiation.cc:407] Connection header received
0619 02:34:16.206787 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:16.206791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:16.206869 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:16.206924 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:16.207529 (+ 605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49390: BlockingRecv error: recv got EOF from 127.0.0.1:49390 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
W20260619 02:34:16.222018 7728 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:16.218756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49396 (local address 127.17.96.253:34193)
0619 02:34:16.218906 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:16.218911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:16.220868 (+ 1957us) server_negotiation.cc:407] Connection header received
0619 02:34:16.221113 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:16.221117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:16.221178 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:16.221232 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:16.221880 (+ 648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49396: BlockingRecv error: recv got EOF from 127.0.0.1:49396 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260619 02:34:16.242543 7728 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:16.240775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49408 (local address 127.17.96.253:34193)
0619 02:34:16.240931 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:16.240937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:16.241339 (+ 402us) server_negotiation.cc:407] Connection header received
0619 02:34:16.241572 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:16.241576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:16.241638 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:16.241713 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:16.242408 (+ 695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49408: BlockingRecv error: recv got EOF from 127.0.0.1:49408 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260619 02:34:16.279973 7728 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:16.278632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49414 (local address 127.17.96.253:34193)
0619 02:34:16.278788 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:16.278795 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:16.278994 (+ 199us) server_negotiation.cc:407] Connection header received
0619 02:34:16.279165 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:16.279168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:16.279219 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:16.279296 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:16.279838 (+ 542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49414: BlockingRecv error: recv got EOF from 127.0.0.1:49414 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260619 02:34:16.347488 7728 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:16.346261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49426 (local address 127.17.96.253:34193)
0619 02:34:16.346430 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:16.346435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:16.346579 (+ 144us) server_negotiation.cc:407] Connection header received
0619 02:34:16.346748 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:16.346752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:16.346808 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:16.346901 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:16.347373 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49426: BlockingRecv error: recv got EOF from 127.0.0.1:49426 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260619 02:34:16.479095 7728 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:16.477870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49436 (local address 127.17.96.253:34193)
0619 02:34:16.478023 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:16.478028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:16.478269 (+ 241us) server_negotiation.cc:407] Connection header received
0619 02:34:16.478370 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:16.478376 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:16.478440 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:16.478514 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:16.478947 (+ 433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49436: BlockingRecv error: recv got EOF from 127.0.0.1:49436 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260619 02:34:16.738482 7728 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:16.737326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49450 (local address 127.17.96.253:34193)
0619 02:34:16.737483 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:16.737487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:16.737605 (+ 118us) server_negotiation.cc:407] Connection header received
0619 02:34:16.737730 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:16.737732 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:16.737800 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:16.737877 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:16.738338 (+ 461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49450: BlockingRecv error: recv got EOF from 127.0.0.1:49450 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:34:17.008263 7831 catalog_manager.cc:1259] Loaded cluster ID: 7c1f0a5ae99d45b5b978792e845a4710
I20260619 02:34:17.008344 7831 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1da2735702cb466e941e06432babe553: loading cluster ID for follower catalog manager: success
I20260619 02:34:17.010078 7831 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1da2735702cb466e941e06432babe553: acquiring CA information for follower catalog manager: success
I20260619 02:34:17.011284 7831 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1da2735702cb466e941e06432babe553: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:34:17.066337 7856 catalog_manager.cc:1259] Loaded cluster ID: 7c1f0a5ae99d45b5b978792e845a4710
I20260619 02:34:17.066424 7856 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3fc78b8b166c44e99b2c58bbf49dda4a: loading cluster ID for follower catalog manager: success
I20260619 02:34:17.068076 7856 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3fc78b8b166c44e99b2c58bbf49dda4a: acquiring CA information for follower catalog manager: success
I20260619 02:34:17.069283 7856 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3fc78b8b166c44e99b2c58bbf49dda4a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:34:17.256548 7930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:17.252745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49456 (local address 127.17.96.253:34193)
0619 02:34:17.253019 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:17.253023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:17.253039 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:34:17.253127 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:17.253129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:17.253253 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:17.253350 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:34:17.253613 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:17.254253 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:17.255087 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:17.255309 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:17.255387 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:17.255663 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:34:17.256130 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:34:17.256223 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:17.256281 (+ 58us) server_negotiation.cc:299] Negotiation successful
0619 02:34:17.256334 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":85,"threads_started":1}
I20260619 02:34:17.263509 7936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:17.259783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53838 (local address 127.17.96.252:37263)
0619 02:34:17.260071 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:17.260075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:17.260091 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:34:17.260209 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:17.260212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:17.260330 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:17.260428 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:34:17.260615 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:17.261140 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:17.262007 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:17.262254 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:17.262335 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:17.262591 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:34:17.263103 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:34:17.263200 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:17.263257 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 02:34:17.263310 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":85,"threads_started":1}
I20260619 02:34:17.271309 7930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:17.267069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49458 (local address 127.17.96.253:34193)
0619 02:34:17.267230 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:17.267234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:17.267513 (+ 279us) server_negotiation.cc:407] Connection header received
0619 02:34:17.267726 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:17.267729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:17.267786 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:17.267858 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:34:17.268283 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:17.269056 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:17.270039 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:17.270364 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:17.270496 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:17.270756 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:34:17.270840 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:34:17.270909 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:17.271069 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 02:34:17.271141 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 02:34:17.271421 7936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:17.266839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53852 (local address 127.17.96.252:37263)
0619 02:34:17.267161 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:17.267165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:17.267400 (+ 235us) server_negotiation.cc:407] Connection header received
0619 02:34:17.267641 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:17.267646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:17.267730 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:17.267799 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:34:17.268181 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:17.268981 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:17.270120 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:17.270472 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:17.270582 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:17.270906 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:34:17.270995 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:34:17.271088 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:17.271230 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 02:34:17.271282 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204}
I20260619 02:34:17.271646 7942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:17.266734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55502 (local address 127.17.96.254:46021)
0619 02:34:17.267022 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:17.267027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:17.267263 (+ 236us) server_negotiation.cc:407] Connection header received
0619 02:34:17.267376 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:17.267382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:17.267571 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:17.267733 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:34:17.267993 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:17.268653 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:17.269832 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:17.270190 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:17.270287 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:17.270629 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:34:17.271040 (+ 411us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:34:17.271346 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:17.271376 (+ 30us) server_negotiation.cc:299] Negotiation successful
0619 02:34:17.271473 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":91,"threads_started":1}
I20260619 02:34:17.275348 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 7579
I20260619 02:34:17.282058 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 7654
I20260619 02:34:17.288374 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 7729
2026-06-19T02:34:17Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3189 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:34:17 dist-test-slave-4qw6 krb5kdc[7947](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:34:17 dist-test-slave-4qw6 krb5kdc[7947](info): set up 2 sockets
Jun 19 02:34:17 dist-test-slave-4qw6 krb5kdc[7947](info): commencing operation
krb5kdc: starting...
W20260619 02:34:18.330349 17795 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/test-user.keytab.
Jun 19 02:34:18 dist-test-slave-4qw6 krb5kdc[7947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:34:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:34:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.96.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
Entry for principal kudu/127.17.96.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab.
I20260619 02:34:18.457259 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:44767
--webserver_interface=127.17.96.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:35533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:44767,127.17.96.253:42311,127.17.96.252:35977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:34:18.572410 7960 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:34:18.572822 7960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:34:18.572901 7960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:34:18.572981 7960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:34:18.577270 7960 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:34:18.577399 7960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:34:18.577438 7960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:34:18.577476 7960 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:34:18.577509 7960 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:34:18.583305 7960 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:35533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:44767,127.17.96.253:42311,127.17.96.252:35977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.254.keytab
--principal=kudu/127.17.96.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.254:44767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.96.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:34:18.584434 7960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:34:18.585443 7960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:34:18.594282 7968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:18.594301 7965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:34:18.594473 7960 server_base.cc:1034] running on GCE node
W20260619 02:34:18.594645 7966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:34:18.595539 7960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:34:18.596828 7960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:34:18.600138 7960 hybrid_clock.cc:648] HybridClock initialized: now 1781836458600089 us; error 60 us; skew 500 ppm
Jun 19 02:34:18 dist-test-slave-4qw6 krb5kdc[7947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836458, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:34:18.607584 7960 init.cc:377] Logged in from keytab as kudu/127.17.96.254@KRBTEST.COM (short username kudu)
I20260619 02:34:18.608651 7960 webserver.cc:460] Webserver started at http://127.17.96.254:45867/ using document root <none> and password file <none>
I20260619 02:34:18.609277 7960 fs_manager.cc:359] Metadata directory not provided
I20260619 02:34:18.609385 7960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:34:18.609634 7960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:34:18.611871 7960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-0/data/instance:
uuid: "33ad273cbbd8448f9ac746777e5330d2"
format_stamp: "Formatted at 2026-06-19 02:34:18 on dist-test-slave-4qw6"
server_key: "658474e59ed1d00b2ca0417d689707ca"
server_key_iv: "abb329b881c5c16e7a46427e99c25c00"
server_key_version: "encryptionkey@0"
I20260619 02:34:18.612493 7960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-0/wal/instance:
uuid: "33ad273cbbd8448f9ac746777e5330d2"
format_stamp: "Formatted at 2026-06-19 02:34:18 on dist-test-slave-4qw6"
server_key: "658474e59ed1d00b2ca0417d689707ca"
server_key_iv: "abb329b881c5c16e7a46427e99c25c00"
server_key_version: "encryptionkey@0"
I20260619 02:34:18.616640 7960 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 02:34:18.619346 7975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:34:18.620688 7960 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 02:34:18.620849 7960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-0/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-0/wal
uuid: "33ad273cbbd8448f9ac746777e5330d2"
format_stamp: "Formatted at 2026-06-19 02:34:18 on dist-test-slave-4qw6"
server_key: "658474e59ed1d00b2ca0417d689707ca"
server_key_iv: "abb329b881c5c16e7a46427e99c25c00"
server_key_version: "encryptionkey@0"
I20260619 02:34:18.620983 7960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:34:18.639700 7960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:34:18.643428 7960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:34:18.643669 7960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:34:18.651192 7960 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.254:44767
I20260619 02:34:18.651494 8027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.254:44767 every 8 connection(s)
I20260619 02:34:18.652275 7960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-0/data/info.pb
I20260619 02:34:18.654436 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 7960
I20260619 02:34:18.654583 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-0/wal/instance
I20260619 02:34:18.654913 17795 external_mini_cluster.cc:1428] Setting key 4fae5ecfb4fbfa21068a6b5742bd2de0
I20260619 02:34:18.655666 8028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:34:18.661345 8028 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44767 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.96.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.253@KRBTEST.COM" created.
Jun 19 02:34:18 dist-test-slave-4qw6 krb5kdc[7947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836458, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:34:18.678859 8032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:18.665281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58884 (local address 127.17.96.254:44767)
0619 02:34:18.665567 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:18.665578 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:18.665620 (+ 42us) server_negotiation.cc:407] Connection header received
0619 02:34:18.666717 (+ 1097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:18.666721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:18.666964 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:18.667173 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:18.667989 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:18.669081 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:18.670337 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:18.671035 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:18.671237 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:18.673647 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:18.673656 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:18.673672 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:18.673700 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:18.675649 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:18.676151 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:18.676160 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:18.676165 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:18.676290 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:18.676581 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:18.676583 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:18.676584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:18.677080 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:18.677193 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:18.677611 (+ 418us) server_negotiation.cc:299] Negotiation successful
0619 02:34:18.677900 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"spinlock_wait_cycles":1792,"thread_start_us":91,"threads_started":1}
I20260619 02:34:18.678866 8031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:18.664427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:44767 (local address 127.0.0.1:58884)
0619 02:34:18.665039 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:18.665160 (+ 121us) client_negotiation.cc:174] Beginning negotiation
0619 02:34:18.666316 (+ 1156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:34:18.667200 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:34:18.667205 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:34:18.667547 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:34:18.667821 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:18.667834 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:18.669222 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:18.669228 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:18.670214 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:18.670224 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:18.671203 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:18.671205 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:18.671288 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:18.671933 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:34:18.671952 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:34:18.673474 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:34:18.675756 (+ 2282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:18.675765 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:18.675774 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:18.676010 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:18.676375 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:18.676377 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:18.676379 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:18.676497 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:18.677186 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:34:18.677191 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:34:18.677323 (+ 132us) client_negotiation.cc:769] Sending connection context
0619 02:34:18.677480 (+ 157us) client_negotiation.cc:240] Negotiation successful
0619 02:34:18.678020 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":149,"spinlock_wait_cycles":6016,"thread_start_us":151,"threads_started":1}
I20260619 02:34:18.681517 8028 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.253.keytab.
Entry for principal kudu/127.17.96.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.253.keytab.
I20260619 02:34:18.682425 8031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:18.682097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:42311 (local address 127.0.0.1:60216)
0619 02:34:18.682251 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:18.682335 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.253:42311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260619 02:34:18.682655 7978 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.96.253:42311: connect: Connection refused (error 111)
I20260619 02:34:18.683457 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.253:42311
--webserver_interface=127.17.96.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.253.keytab
--principal=kudu/127.17.96.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:35533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:44767,127.17.96.253:42311,127.17.96.252:35977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:34:18.683610 8031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:18.683345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:42311 (local address 127.0.0.1:60232)
0619 02:34:18.683453 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:18.683512 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.253:42311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 02:34:18.683802 8028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.253:42311: Network error: Client connection negotiation failed: client connection to 127.17.96.253:42311: connect: Connection refused (error 111)
I20260619 02:34:18.733098 8028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } attempt: 1
I20260619 02:34:18.734082 8031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:18.733731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:42311 (local address 127.0.0.1:60246)
0619 02:34:18.733921 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:18.733994 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.253:42311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260619 02:34:18.734907 8031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:18.734692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:42311 (local address 127.0.0.1:60258)
0619 02:34:18.734802 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:18.734849 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.253:42311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 02:34:18.735122 8028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.253:42311: Network error: Client connection negotiation failed: client connection to 127.17.96.253:42311: connect: Connection refused (error 111)
W20260619 02:34:18.798807 8035 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:34:18.799124 8035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:34:18.799192 8035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:34:18.799295 8035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 02:34:18.803321 8028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } attempt: 2
W20260619 02:34:18.803550 8035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:34:18.803619 8035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:34:18.803660 8035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:34:18.803694 8035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:34:18.803726 8035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:34:18.804247 8031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:18.803938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:42311 (local address 127.0.0.1:60266)
0619 02:34:18.804103 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:18.804172 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.253:42311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260619 02:34:18.804937 8031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:18.804722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:42311 (local address 127.0.0.1:60282)
0619 02:34:18.804828 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:18.804881 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.253:42311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260619 02:34:18.805051 8028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.253:42311: Network error: Client connection negotiation failed: client connection to 127.17.96.253:42311: connect: Connection refused (error 111)
I20260619 02:34:18.809728 8035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:35533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:44767,127.17.96.253:42311,127.17.96.252:35977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.253.keytab
--principal=kudu/127.17.96.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.253:42311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.96.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:34:18.810994 8035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:34:18.812037 8035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:34:18.820698 8040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:18.820680 8043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:18.820680 8041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:34:18.821121 8035 server_base.cc:1034] running on GCE node
I20260619 02:34:18.821902 8035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:34:18.823266 8035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:34:18.824447 8035 hybrid_clock.cc:648] HybridClock initialized: now 1781836458824432 us; error 35 us; skew 500 ppm
Jun 19 02:34:18 dist-test-slave-4qw6 krb5kdc[7947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836458, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:34:18.827483 8035 init.cc:377] Logged in from keytab as kudu/127.17.96.253@KRBTEST.COM (short username kudu)
I20260619 02:34:18.828419 8035 webserver.cc:460] Webserver started at http://127.17.96.253:39897/ using document root <none> and password file <none>
I20260619 02:34:18.828985 8035 fs_manager.cc:359] Metadata directory not provided
I20260619 02:34:18.829047 8035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:34:18.829226 8035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:34:18.831476 8035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-1/data/instance:
uuid: "61cbba2b62ef4ebd8bce5f1be99b4753"
format_stamp: "Formatted at 2026-06-19 02:34:18 on dist-test-slave-4qw6"
server_key: "2f4747a976e7c85315605e6ce0ef3e8c"
server_key_iv: "3f5340b868e38ac43912b477ddb51f09"
server_key_version: "encryptionkey@0"
I20260619 02:34:18.832033 8035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-1/wal/instance:
uuid: "61cbba2b62ef4ebd8bce5f1be99b4753"
format_stamp: "Formatted at 2026-06-19 02:34:18 on dist-test-slave-4qw6"
server_key: "2f4747a976e7c85315605e6ce0ef3e8c"
server_key_iv: "3f5340b868e38ac43912b477ddb51f09"
server_key_version: "encryptionkey@0"
I20260619 02:34:18.836035 8035 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:34:18.838668 8050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:34:18.840134 8035 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:34:18.840284 8035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-1/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-1/wal
uuid: "61cbba2b62ef4ebd8bce5f1be99b4753"
format_stamp: "Formatted at 2026-06-19 02:34:18 on dist-test-slave-4qw6"
server_key: "2f4747a976e7c85315605e6ce0ef3e8c"
server_key_iv: "3f5340b868e38ac43912b477ddb51f09"
server_key_version: "encryptionkey@0"
I20260619 02:34:18.840407 8035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:34:18.860904 8035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:34:18.864334 8035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:34:18.864553 8035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:34:18.871860 8035 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.253:42311
I20260619 02:34:18.872175 8102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.253:42311 every 8 connection(s)
I20260619 02:34:18.872906 8035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-1/data/info.pb
I20260619 02:34:18.875844 8103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:34:18.879980 8103 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44767 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:34:18.880654 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 8035
I20260619 02:34:18.880789 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-1/wal/instance
I20260619 02:34:18.881086 17795 external_mini_cluster.cc:1428] Setting key 056d6d835ccde2793f4a7446cac514a6
WARNING: no policy specified for kudu/127.17.96.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.96.252@KRBTEST.COM" created.
Jun 19 02:34:18 dist-test-slave-4qw6 krb5kdc[7947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836458, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.253@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:34:18.896219 8028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } attempt: 3
I20260619 02:34:18.898310 8032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:18.882293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58892 (local address 127.17.96.254:44767)
0619 02:34:18.882483 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:18.882490 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:18.883440 (+ 950us) server_negotiation.cc:407] Connection header received
0619 02:34:18.884647 (+ 1207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:18.884653 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:18.884726 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:18.884828 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:18.886197 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:18.887149 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:18.888381 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:18.888809 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:18.888952 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:18.893635 (+ 4683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:18.893638 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:18.893641 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:18.893671 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:18.895424 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:18.896030 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:18.896034 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:18.896036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:18.896125 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:18.896574 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:18.896581 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:18.896586 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:18.896882 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:18.897003 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:18.897894 (+ 891us) server_negotiation.cc:299] Negotiation successful
0619 02:34:18.898100 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":61}
I20260619 02:34:18.899613 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:18.882670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:44767 (local address 127.0.0.1:58892)
0619 02:34:18.883229 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:18.883294 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 02:34:18.884416 (+ 1122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:34:18.885002 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:34:18.885008 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:34:18.885408 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:34:18.886025 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:18.886041 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:18.887322 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:18.887326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:18.888235 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:18.888246 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:18.888994 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:18.888997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:18.889145 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:18.889883 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:34:18.889900 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:34:18.893402 (+ 3502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:34:18.895569 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:18.895576 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:18.895594 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:18.895884 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:18.896223 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:18.896226 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:18.896228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:18.896420 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:18.897088 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:34:18.897097 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:34:18.897485 (+ 388us) client_negotiation.cc:769] Sending connection context
0619 02:34:18.897985 (+ 500us) client_negotiation.cc:240] Negotiation successful
0619 02:34:18.898412 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":134,"threads_started":1}
I20260619 02:34:18.901515 8103 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:34:18 dist-test-slave-4qw6 krb5kdc[7947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836458, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for kudu/127.17.96.253@KRBTEST.COM
Jun 19 02:34:18 dist-test-slave-4qw6 krb5kdc[7947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836458, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.253@KRBTEST.COM for kudu/127.17.96.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.96.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.252.keytab.
Entry for principal kudu/127.17.96.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.252.keytab.
I20260619 02:34:18.908878 8031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:18.896722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:42311 (local address 127.0.0.1:60286)
0619 02:34:18.896919 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:18.896934 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:34:18.897018 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:34:18.897682 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:34:18.897685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:34:18.897900 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:34:18.898522 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:18.898532 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:18.899597 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:18.899603 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:18.900386 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:18.900397 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:18.900992 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:18.900996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:18.901112 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:18.901980 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:34:18.902001 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:34:18.904203 (+ 2202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:34:18.906824 (+ 2621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:18.906827 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:18.906828 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:18.907100 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:18.907437 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:18.907440 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:18.907442 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:18.907537 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:18.908250 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:34:18.908254 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:34:18.908373 (+ 119us) client_negotiation.cc:769] Sending connection context
0619 02:34:18.908490 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 02:34:18.908663 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":2}
I20260619 02:34:18.909029 17795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
/tmp/dist-test-taskhpFBxW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.252:35977
--webserver_interface=127.17.96.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.252.keytab
--principal=kudu/127.17.96.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.96.212:35533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.96.254:44767,127.17.96.253:42311,127.17.96.252:35977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:34:18.909170 8108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:18.896829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60286 (local address 127.17.96.253:42311)
0619 02:34:18.897303 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:18.897315 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:18.897335 (+ 20us) server_negotiation.cc:407] Connection header received
0619 02:34:18.897370 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:18.897374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:18.897551 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:18.897702 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:18.898650 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:18.899475 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:18.900491 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:18.900868 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:18.901001 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:18.904335 (+ 3334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:18.904345 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:18.904361 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:18.904392 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:18.906657 (+ 2265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:18.907240 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:18.907247 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:18.907248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:18.907329 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:18.907716 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:18.907719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:18.907721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:18.908088 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:18.908273 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:18.908597 (+ 324us) server_negotiation.cc:299] Negotiation successful
0619 02:34:18.908796 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":161,"threads_started":1}
I20260619 02:34:18.911077 8028 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:34:18.911836 8031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:18.911583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35977 (local address 127.0.0.1:56838)
0619 02:34:18.911708 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:18.911761 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 02:34:18.912338 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:18.902127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:42311 (local address 127.0.0.1:60302)
0619 02:34:18.902307 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:18.902321 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:34:18.902397 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:34:18.902672 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:34:18.902675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:34:18.902883 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:34:18.902979 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:18.902984 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:18.903905 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:18.903911 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:18.904508 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:18.904516 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:18.904970 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:18.904973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:18.905106 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:18.905701 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:34:18.905718 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:34:18.907312 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:34:18.910618 (+ 3306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:18.910623 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:18.910628 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:18.910881 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:18.911185 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:18.911190 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:18.911193 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:18.911284 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:18.911758 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:34:18.911762 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:34:18.911847 (+ 85us) client_negotiation.cc:769] Sending connection context
0619 02:34:18.911957 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 02:34:18.912115 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":24,"spinlock_wait_cycles":8064}
I20260619 02:34:18.912783 8031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:18.912317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35977 (local address 127.0.0.1:56840)
0619 02:34:18.912648 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:18.912702 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":120,"threads_started":1}
W20260619 02:34:18.912925 8028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.252:35977: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35977: connect: Connection refused (error 111)
I20260619 02:34:18.912941 8109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:18.902201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60302 (local address 127.17.96.253:42311)
0619 02:34:18.902471 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:18.902477 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:18.902488 (+ 11us) server_negotiation.cc:407] Connection header received
0619 02:34:18.902517 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:18.902519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:18.902554 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:18.902663 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:18.903094 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:18.903795 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:18.904629 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:18.904864 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:18.904973 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:18.907470 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:18.907473 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:18.907475 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:18.907494 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:18.910483 (+ 2989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:18.910992 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:18.910997 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:18.911000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:18.911088 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:18.911378 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:18.911382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:18.911385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:18.911627 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:18.911722 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:18.912559 (+ 837us) server_negotiation.cc:299] Negotiation successful
0619 02:34:18.912691 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":218,"thread_start_us":111,"threads_started":1}
I20260619 02:34:18.913568 8103 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:34:18.914333 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:18.914008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35977 (local address 127.0.0.1:56846)
0619 02:34:18.914152 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:18.914247 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260619 02:34:18.914613 8054 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35977: connect: Connection refused (error 111)
I20260619 02:34:18.915535 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:18.915250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35977 (local address 127.0.0.1:56860)
0619 02:34:18.915391 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:18.915455 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260619 02:34:18.915691 8103 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.252:35977: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35977: connect: Connection refused (error 111)
I20260619 02:34:18.944103 8028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } attempt: 1
I20260619 02:34:18.945164 8111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:18.944741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35977 (local address 127.0.0.1:56864)
0619 02:34:18.944988 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:18.945055 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260619 02:34:18.946069 8111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:18.945837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35977 (local address 127.0.0.1:56870)
0619 02:34:18.945947 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:18.946009 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 02:34:18.946230 8028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.252:35977: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35977: connect: Connection refused (error 111)
I20260619 02:34:18.965015 8103 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } attempt: 1
I20260619 02:34:18.966056 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:18.965682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35977 (local address 127.0.0.1:56880)
0619 02:34:18.965891 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:18.965956 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260619 02:34:18.966823 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:18.966628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35977 (local address 127.0.0.1:56894)
0619 02:34:18.966726 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:18.966771 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 02:34:18.967032 8103 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.252:35977: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35977: connect: Connection refused (error 111)
I20260619 02:34:19.021438 8028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } attempt: 2
I20260619 02:34:19.022502 8111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:19.022104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35977 (local address 127.0.0.1:56908)
0619 02:34:19.022301 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:19.022399 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260619 02:34:19.023350 8111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:19.023047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35977 (local address 127.0.0.1:56912)
0619 02:34:19.023185 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:19.023252 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260619 02:34:19.023556 8028 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.252:35977: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35977: connect: Connection refused (error 111)
W20260619 02:34:19.028643 8110 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:34:19.028972 8110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:34:19.029001 8110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:34:19.029047 8110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:34:19.033286 8110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:34:19.033387 8110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:34:19.033412 8110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:34:19.033433 8110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:34:19.033452 8110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:34:19.035202 8103 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } attempt: 2
I20260619 02:34:19.036130 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:19.035792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35977 (local address 127.0.0.1:56922)
0619 02:34:19.035978 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:19.036046 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260619 02:34:19.036998 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:19.036750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35977 (local address 127.0.0.1:56934)
0619 02:34:19.036887 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:19.036937 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260619 02:34:19.037214 8103 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.96.252:35977: Network error: Client connection negotiation failed: client connection to 127.17.96.252:35977: connect: Connection refused (error 111)
I20260619 02:34:19.039898 8110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.96.212:35533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.96.254:44767,127.17.96.253:42311,127.17.96.252:35977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/krb5kdc/kudu_127.17.96.252.keytab
--principal=kudu/127.17.96.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.96.252:35977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.96.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 01:50:15 UTC on ccycloud.quasar-qviwlx.root.comops.site
build id 4454979
I20260619 02:34:19.041098 8110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:34:19.042295 8110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:34:19.051870 8110 server_base.cc:1034] running on GCE node
W20260619 02:34:19.051828 8118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:19.051828 8117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:34:19.051820 8120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:34:19.052812 8110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:34:19.054104 8110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:34:19.055267 8110 hybrid_clock.cc:648] HybridClock initialized: now 1781836459055251 us; error 32 us; skew 500 ppm
Jun 19 02:34:19 dist-test-slave-4qw6 krb5kdc[7947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836459, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:34:19.058210 8110 init.cc:377] Logged in from keytab as kudu/127.17.96.252@KRBTEST.COM (short username kudu)
I20260619 02:34:19.059155 8110 webserver.cc:460] Webserver started at http://127.17.96.252:42251/ using document root <none> and password file <none>
I20260619 02:34:19.059752 8110 fs_manager.cc:359] Metadata directory not provided
I20260619 02:34:19.059850 8110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:34:19.060096 8110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:34:19.062335 8110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-2/data/instance:
uuid: "af2905691054452ab6410fa1c440507b"
format_stamp: "Formatted at 2026-06-19 02:34:19 on dist-test-slave-4qw6"
server_key: "a322e9f911e46c1f0e022e361e725c0e"
server_key_iv: "3250387c7e41e7c938412c7dc48ad589"
server_key_version: "encryptionkey@0"
I20260619 02:34:19.062922 8110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-2/wal/instance:
uuid: "af2905691054452ab6410fa1c440507b"
format_stamp: "Formatted at 2026-06-19 02:34:19 on dist-test-slave-4qw6"
server_key: "a322e9f911e46c1f0e022e361e725c0e"
server_key_iv: "3250387c7e41e7c938412c7dc48ad589"
server_key_version: "encryptionkey@0"
I20260619 02:34:19.066869 8110 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 02:34:19.069482 8127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:34:19.070822 8110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:34:19.070981 8110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-2/data,/tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-2/wal
uuid: "af2905691054452ab6410fa1c440507b"
format_stamp: "Formatted at 2026-06-19 02:34:19 on dist-test-slave-4qw6"
server_key: "a322e9f911e46c1f0e022e361e725c0e"
server_key_iv: "3250387c7e41e7c938412c7dc48ad589"
server_key_version: "encryptionkey@0"
I20260619 02:34:19.071115 8110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:34:19.088943 8110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:34:19.092574 8110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:34:19.092805 8110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:34:19.100291 8110 rpc_server.cc:307] RPC server started. Bound to: 127.17.96.252:35977
I20260619 02:34:19.100615 8179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.96.252:35977 every 8 connection(s)
I20260619 02:34:19.101357 8110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-2/data/info.pb
I20260619 02:34:19.104350 8180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:34:19.108101 17795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu as pid 8110
I20260619 02:34:19.108242 17795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskhpFBxW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781836162001484-17795-0/minicluster-data/master-2/wal/instance
I20260619 02:34:19.108595 17795 external_mini_cluster.cc:1428] Setting key 8908c3d33bce46352428041c34587624
I20260619 02:34:19.108529 8180 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44767 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:34:19 dist-test-slave-4qw6 krb5kdc[7947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836459, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.252@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
Jun 19 02:34:19 dist-test-slave-4qw6 krb5kdc[7947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.252@KRBTEST.COM
I20260619 02:34:19.122725 8028 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } attempt: 3
I20260619 02:34:19.124507 8032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:19.110739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58908 (local address 127.17.96.254:44767)
0619 02:34:19.110882 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.110886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.111187 (+ 301us) server_negotiation.cc:407] Connection header received
0619 02:34:19.111692 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.111696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.111745 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.111849 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.113021 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.113846 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.115244 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.115595 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.115710 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:19.118769 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:19.118776 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:19.118781 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:19.118818 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:19.120561 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:19.121349 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:19.121355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:19.121360 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:19.121457 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:19.121937 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:19.121941 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:19.121942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:19.122235 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:19.122331 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:19.124103 (+ 1772us) server_negotiation.cc:299] Negotiation successful
0619 02:34:19.124256 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 02:34:19.125489 8183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:19.110076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56936 (local address 127.17.96.252:35977)
0619 02:34:19.110530 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.110559 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.110600 (+ 41us) server_negotiation.cc:407] Connection header received
0619 02:34:19.111473 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.111492 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.111750 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.111923 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.112851 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.114111 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.115276 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.115858 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.115965 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:19.119228 (+ 3263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:19.119234 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:19.119246 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:19.119292 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:19.121319 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:19.121875 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:19.121883 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:19.121885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:19.121965 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:19.122250 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:19.122252 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:19.122253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:19.122808 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:19.123022 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:19.123892 (+ 870us) server_negotiation.cc:299] Negotiation successful
0619 02:34:19.124316 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":135,"server-negotiator.queue_time_us":285,"spinlock_wait_cycles":17792,"thread_start_us":145,"threads_started":1}
I20260619 02:34:19.125488 8184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:19.110597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.254:44767 (local address 127.0.0.1:58908)
0619 02:34:19.111037 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:19.111089 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 02:34:19.111496 (+ 407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:34:19.111972 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:34:19.111979 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:34:19.112475 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:34:19.112878 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:19.112892 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.113956 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.113959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:19.115091 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:19.115102 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.115748 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.115751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:19.115969 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:19.116623 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:34:19.116651 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:34:19.118561 (+ 1910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:34:19.120712 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:19.120724 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:19.120740 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:19.121244 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:19.121579 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:19.121582 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:19.121584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:19.121741 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:19.122347 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:34:19.122354 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:34:19.122777 (+ 423us) client_negotiation.cc:769] Sending connection context
0619 02:34:19.124001 (+ 1224us) client_negotiation.cc:240] Negotiation successful
0619 02:34:19.124882 (+ 881us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":663,"thread_start_us":134,"threads_started":1}
I20260619 02:34:19.127907 8180 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:34:19.128367 8103 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } attempt: 3
Jun 19 02:34:19 dist-test-slave-4qw6 krb5kdc[7947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836458, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.254@KRBTEST.COM for kudu/127.17.96.252@KRBTEST.COM
I20260619 02:34:19.134640 8111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:19.123233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35977 (local address 127.0.0.1:56952)
0619 02:34:19.123349 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:19.123365 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:34:19.123459 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:34:19.124332 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:34:19.124335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:34:19.124538 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:34:19.124683 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:19.124691 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.125587 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.125593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:19.126496 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:19.126507 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.127108 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.127112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:19.127209 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:19.128035 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:34:19.128072 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:34:19.129827 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:34:19.132781 (+ 2954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:19.132786 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:19.132789 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:19.133097 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:19.133445 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:19.133451 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:19.133456 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:19.133558 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:19.134028 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:34:19.134032 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:34:19.134145 (+ 113us) client_negotiation.cc:769] Sending connection context
0619 02:34:19.134264 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 02:34:19.134439 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 02:34:19.134689 8185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:19.123530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56952 (local address 127.17.96.252:35977)
0619 02:34:19.124005 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.124017 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.124030 (+ 13us) server_negotiation.cc:407] Connection header received
0619 02:34:19.124066 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.124070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.124113 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.124288 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.124808 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.125458 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.126623 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.126958 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.127068 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:19.129974 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:19.129978 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:19.129980 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:19.130004 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:19.132625 (+ 2621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:19.133234 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:19.133239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:19.133241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:19.133338 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:19.133667 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:19.133670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:19.133672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:19.133939 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:19.134046 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:19.134343 (+ 297us) server_negotiation.cc:299] Negotiation successful
0619 02:34:19.134480 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":123,"threads_started":1}
Jun 19 02:34:19 dist-test-slave-4qw6 krb5kdc[7947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836458, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.253@KRBTEST.COM for kudu/127.17.96.252@KRBTEST.COM
Jun 19 02:34:19 dist-test-slave-4qw6 krb5kdc[7947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836459, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.252@KRBTEST.COM for kudu/127.17.96.253@KRBTEST.COM
Jun 19 02:34:19 dist-test-slave-4qw6 krb5kdc[7947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.254@KRBTEST.COM
I20260619 02:34:19.139014 8028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2: Bootstrap starting.
I20260619 02:34:19.141048 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:19.128878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35977 (local address 127.0.0.1:56960)
0619 02:34:19.129011 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:19.129026 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:34:19.129101 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:34:19.130097 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:34:19.130100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:34:19.130292 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:34:19.130409 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:19.130417 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.131548 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.131551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:19.132385 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:19.132396 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.133615 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.133618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:19.133711 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:19.134230 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:34:19.134248 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:34:19.135826 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:34:19.137906 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:19.137910 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:19.137912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:19.138195 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:19.139722 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:19.139725 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:19.139727 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:19.139830 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:19.140339 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:34:19.140358 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:34:19.140484 (+ 126us) client_negotiation.cc:769] Sending connection context
0619 02:34:19.140679 (+ 195us) client_negotiation.cc:240] Negotiation successful
0619 02:34:19.140852 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":1}
I20260619 02:34:19.141335 8109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:19.128698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60304 (local address 127.17.96.253:42311)
0619 02:34:19.129026 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.129029 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.129039 (+ 10us) server_negotiation.cc:407] Connection header received
0619 02:34:19.129078 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.129080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.129116 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.129208 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.129670 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.130824 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.133540 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.134389 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.134471 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:19.137241 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:19.137248 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:19.137250 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:19.137276 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:19.139407 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:19.139935 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:19.139939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:19.139941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:19.140028 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:19.140338 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:19.140341 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:19.140343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:19.140631 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:19.140720 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:19.141070 (+ 350us) server_negotiation.cc:299] Negotiation successful
0619 02:34:19.141197 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275}
I20260619 02:34:19.141721 8032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:19.128421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58924 (local address 127.17.96.254:44767)
0619 02:34:19.129632 (+ 1211us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.129636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.129650 (+ 14us) server_negotiation.cc:407] Connection header received
0619 02:34:19.129683 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.129686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.129731 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.129854 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.130750 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.131496 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.132874 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.133246 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.133352 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:19.137944 (+ 4592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:19.137949 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:19.137955 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:19.137988 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:19.139599 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:19.140156 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:19.140160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:19.140162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:19.140249 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:19.140585 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:19.140590 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:19.140592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:19.140836 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:19.141017 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:19.141419 (+ 402us) server_negotiation.cc:299] Negotiation successful
0619 02:34:19.141564 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1129}
I20260619 02:34:19.141754 8183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:19.129603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56960 (local address 127.17.96.252:35977)
0619 02:34:19.129880 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.129884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.129896 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:34:19.129929 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.129933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.129970 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.130528 (+ 558us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.130612 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.131415 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.132536 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.133482 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.134006 (+ 524us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:19.136021 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:19.136026 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:19.136030 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:19.136062 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:19.137747 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:19.138595 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:19.138599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:19.138602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:19.139591 (+ 989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:19.139962 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:19.139966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:19.139968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:19.140222 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:19.141395 (+ 1173us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:19.141464 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 02:34:19.141598 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":89,"server-negotiator.queue_time_us":200}
I20260619 02:34:19.142653 8184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:19.128404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.253:42311 (local address 127.0.0.1:60304)
0619 02:34:19.128700 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:19.128714 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:34:19.128795 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:34:19.129223 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:34:19.129226 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:34:19.129402 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:34:19.129527 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:19.129533 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.130996 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.131001 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:19.133414 (+ 2413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:19.133424 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.134628 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.134631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:19.134740 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:19.135159 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:34:19.135175 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:34:19.137079 (+ 1904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:34:19.139532 (+ 2453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:19.139535 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:19.139537 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:19.139809 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:19.140140 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:19.140143 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:19.140145 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:19.140230 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:19.140791 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:34:19.140794 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:34:19.140892 (+ 98us) client_negotiation.cc:769] Sending connection context
0619 02:34:19.142313 (+ 1421us) client_negotiation.cc:240] Negotiation successful
0619 02:34:19.142480 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":77}
I20260619 02:34:19.142988 8028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2: Neither blocks nor log segments found. Creating new log.
I20260619 02:34:19.143635 8180 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:34:19.144097 8028 log.cc:824] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2: Log is configured to *not* fsync() on all Append() calls
I20260619 02:34:19.145264 8103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61cbba2b62ef4ebd8bce5f1be99b4753: Bootstrap starting.
I20260619 02:34:19.147349 8028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2: No bootstrap required, opened a new log
I20260619 02:34:19.148038 8103 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61cbba2b62ef4ebd8bce5f1be99b4753: Neither blocks nor log segments found. Creating new log.
I20260619 02:34:19.149022 8103 log.cc:824] T 00000000000000000000000000000000 P 61cbba2b62ef4ebd8bce5f1be99b4753: Log is configured to *not* fsync() on all Append() calls
Jun 19 02:34:19 dist-test-slave-4qw6 krb5kdc[7947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.96.253@KRBTEST.COM
I20260619 02:34:19.151525 8103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61cbba2b62ef4ebd8bce5f1be99b4753: No bootstrap required, opened a new log
I20260619 02:34:19.151813 8028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ad273cbbd8448f9ac746777e5330d2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44767 } } peers { permanent_uuid: "61cbba2b62ef4ebd8bce5f1be99b4753" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } } peers { permanent_uuid: "af2905691054452ab6410fa1c440507b" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } }
I20260619 02:34:19.152091 8028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33ad273cbbd8448f9ac746777e5330d2, State: Initialized, Role: FOLLOWER
I20260619 02:34:19.152459 8028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ad273cbbd8448f9ac746777e5330d2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44767 } } peers { permanent_uuid: "61cbba2b62ef4ebd8bce5f1be99b4753" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } } peers { permanent_uuid: "af2905691054452ab6410fa1c440507b" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } }
I20260619 02:34:19.153012 8189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ad273cbbd8448f9ac746777e5330d2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44767 } } peers { permanent_uuid: "61cbba2b62ef4ebd8bce5f1be99b4753" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } } peers { permanent_uuid: "af2905691054452ab6410fa1c440507b" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } } }
I20260619 02:34:19.153266 8189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:34:19.153919 8028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2 [sys.catalog]: configured and running, proceeding with master startup.
Jun 19 02:34:19 dist-test-slave-4qw6 krb5kdc[7947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781836459, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.96.252@KRBTEST.COM for kudu/127.17.96.252@KRBTEST.COM
I20260619 02:34:19.155917 8109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:19.143714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60314 (local address 127.17.96.253:42311)
0619 02:34:19.144760 (+ 1046us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.144764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.144776 (+ 12us) server_negotiation.cc:407] Connection header received
0619 02:34:19.144809 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.144812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.144853 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.144927 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.145894 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.146675 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.147727 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.148048 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.148248 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:19.151348 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:19.151352 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:19.151355 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:19.151383 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:19.153229 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:19.154296 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:19.154303 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:19.154308 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:19.154411 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:19.154757 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:19.154761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:19.154762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:19.155012 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:19.155211 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:19.155590 (+ 379us) server_negotiation.cc:299] Negotiation successful
0619 02:34:19.155721 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":973}
I20260619 02:34:19.156318 8103 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61cbba2b62ef4ebd8bce5f1be99b4753 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ad273cbbd8448f9ac746777e5330d2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44767 } } peers { permanent_uuid: "61cbba2b62ef4ebd8bce5f1be99b4753" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } } peers { permanent_uuid: "af2905691054452ab6410fa1c440507b" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } }
I20260619 02:34:19.156606 8103 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61cbba2b62ef4ebd8bce5f1be99b4753 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61cbba2b62ef4ebd8bce5f1be99b4753, State: Initialized, Role: FOLLOWER
I20260619 02:34:19.156947 8103 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61cbba2b62ef4ebd8bce5f1be99b4753 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ad273cbbd8448f9ac746777e5330d2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44767 } } peers { permanent_uuid: "61cbba2b62ef4ebd8bce5f1be99b4753" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } } peers { permanent_uuid: "af2905691054452ab6410fa1c440507b" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } }
I20260619 02:34:19.158424 8103 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61cbba2b62ef4ebd8bce5f1be99b4753 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:34:19.160521 8186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:19.144147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.96.252:35977 (local address 127.0.0.1:56974)
0619 02:34:19.144470 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0619 02:34:19.144485 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:34:19.144582 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:34:19.145423 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:34:19.145426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:34:19.145602 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:34:19.145720 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:19.145725 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.150972 (+ 5247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.150979 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:19.151842 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:34:19.151858 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.153679 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.153682 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:34:19.153854 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:19.154407 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:34:19.154423 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:34:19.156100 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:34:19.158781 (+ 2681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:19.158784 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:19.158785 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:19.159041 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:19.159360 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:34:19.159363 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:34:19.159366 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:34:19.159455 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:34:19.159867 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:34:19.159870 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:34:19.159955 (+ 85us) client_negotiation.cc:769] Sending connection context
0619 02:34:19.160087 (+ 132us) client_negotiation.cc:240] Negotiation successful
0619 02:34:19.160323 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":121,"thread_start_us":134,"threads_started":1}
I20260619 02:34:19.161005 8185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:19.144615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56974 (local address 127.17.96.252:35977)
0619 02:34:19.144703 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.144708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.144723 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:34:19.144761 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.144765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.144841 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.144925 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.149969 (+ 5044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.150826 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.152317 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.153556 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.153682 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:19.156501 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:19.156505 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:19.156508 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:19.156535 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:19.158677 (+ 2142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:19.159147 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:19.159153 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:19.159155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:19.159230 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:19.159560 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:19.159563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:19.159565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:19.159739 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:19.159872 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:19.160128 (+ 256us) server_negotiation.cc:299] Negotiation successful
0619 02:34:19.160870 (+ 742us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":22}
I20260619 02:34:19.164251 8180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af2905691054452ab6410fa1c440507b: Bootstrap starting.
W20260619 02:34:19.164363 8212 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61cbba2b62ef4ebd8bce5f1be99b4753: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:34:19.164454 8212 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:34:19.165032 17795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260619 02:34:19.165319 8200 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:34:19.165426 8200 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:34:19.165890 8201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61cbba2b62ef4ebd8bce5f1be99b4753 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ad273cbbd8448f9ac746777e5330d2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44767 } } peers { permanent_uuid: "61cbba2b62ef4ebd8bce5f1be99b4753" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } } peers { permanent_uuid: "af2905691054452ab6410fa1c440507b" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } } }
I20260619 02:34:19.166038 8201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61cbba2b62ef4ebd8bce5f1be99b4753 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:34:19.166440 8002 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "33ad273cbbd8448f9ac746777e5330d2"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58924
I20260619 02:34:19.166700 8002 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 02:34:19.166815 8002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:34:19.167162 8180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af2905691054452ab6410fa1c440507b: Neither blocks nor log segments found. Creating new log.
I20260619 02:34:19.159355 7960 master_runner.cc:186] Error getting master registration for 127.17.96.253:42311: OK, instance_id { permanent_uuid: "61cbba2b62ef4ebd8bce5f1be99b4753" instance_seqno: 1781836458869685 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:34:19.167826 7960 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:34:19.168036 8002 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ad273cbbd8448f9ac746777e5330d2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44767 } } peers { permanent_uuid: "61cbba2b62ef4ebd8bce5f1be99b4753" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } } peers { permanent_uuid: "af2905691054452ab6410fa1c440507b" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } }
I20260619 02:34:19.168082 8180 log.cc:824] T 00000000000000000000000000000000 P af2905691054452ab6410fa1c440507b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:34:19.168888 8002 leader_election.cc:288] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2 [CANDIDATE]: Term 1 election: Requested vote from peers 61cbba2b62ef4ebd8bce5f1be99b4753 (127.17.96.253:42311), af2905691054452ab6410fa1c440507b (127.17.96.252:35977)
I20260619 02:34:19.169306 8077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33ad273cbbd8448f9ac746777e5330d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "61cbba2b62ef4ebd8bce5f1be99b4753"
I20260619 02:34:19.169585 8077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61cbba2b62ef4ebd8bce5f1be99b4753 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:34:19.171083 8077 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 61cbba2b62ef4ebd8bce5f1be99b4753 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33ad273cbbd8448f9ac746777e5330d2 in term 1.
I20260619 02:34:19.171506 7978 leader_election.cc:302] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33ad273cbbd8448f9ac746777e5330d2, 61cbba2b62ef4ebd8bce5f1be99b4753; no voters:
I20260619 02:34:19.171823 8189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:34:19.167896 8035 master_runner.cc:186] Error getting master registration for 127.17.96.252:35977: OK, instance_id { permanent_uuid: "af2905691054452ab6410fa1c440507b" instance_seqno: 1781836459098173 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:34:19.172052 8189 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2 [term 1 LEADER]: Becoming Leader. State: Replica: 33ad273cbbd8448f9ac746777e5330d2, State: Running, Role: LEADER
I20260619 02:34:19.172143 8035 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:34:19.172441 8189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ad273cbbd8448f9ac746777e5330d2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44767 } } peers { permanent_uuid: "61cbba2b62ef4ebd8bce5f1be99b4753" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } } peers { permanent_uuid: "af2905691054452ab6410fa1c440507b" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } }
I20260619 02:34:19.174808 8222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33ad273cbbd8448f9ac746777e5330d2. Latest consensus state: current_term: 1 leader_uuid: "33ad273cbbd8448f9ac746777e5330d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ad273cbbd8448f9ac746777e5330d2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44767 } } peers { permanent_uuid: "61cbba2b62ef4ebd8bce5f1be99b4753" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } } peers { permanent_uuid: "af2905691054452ab6410fa1c440507b" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } } }
I20260619 02:34:19.174957 8222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:34:19.175479 8225 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:34:19.177304 8225 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:34:19.180119 8180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af2905691054452ab6410fa1c440507b: No bootstrap required, opened a new log
I20260619 02:34:19.185228 8180 raft_consensus.cc:348] T 00000000000000000000000000000000 P af2905691054452ab6410fa1c440507b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ad273cbbd8448f9ac746777e5330d2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44767 } } peers { permanent_uuid: "61cbba2b62ef4ebd8bce5f1be99b4753" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } } peers { permanent_uuid: "af2905691054452ab6410fa1c440507b" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } }
I20260619 02:34:19.185563 8180 raft_consensus.cc:729] T 00000000000000000000000000000000 P af2905691054452ab6410fa1c440507b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af2905691054452ab6410fa1c440507b, State: Initialized, Role: FOLLOWER
I20260619 02:34:19.186017 8180 consensus_queue.cc:260] T 00000000000000000000000000000000 P af2905691054452ab6410fa1c440507b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ad273cbbd8448f9ac746777e5330d2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44767 } } peers { permanent_uuid: "61cbba2b62ef4ebd8bce5f1be99b4753" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } } peers { permanent_uuid: "af2905691054452ab6410fa1c440507b" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } }
I20260619 02:34:19.187714 8180 sys_catalog.cc:564] T 00000000000000000000000000000000 P af2905691054452ab6410fa1c440507b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:34:19.187762 8154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33ad273cbbd8448f9ac746777e5330d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "af2905691054452ab6410fa1c440507b"
I20260619 02:34:19.188041 8154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af2905691054452ab6410fa1c440507b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:34:19.190551 8154 raft_consensus.cc:2454] T 00000000000000000000000000000000 P af2905691054452ab6410fa1c440507b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33ad273cbbd8448f9ac746777e5330d2 in term 1.
I20260619 02:34:19.191865 8032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:19.171338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58928 (local address 127.17.96.254:44767)
0619 02:34:19.171910 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.171916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.174159 (+ 2243us) server_negotiation.cc:407] Connection header received
0619 02:34:19.174323 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.174326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.174387 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.174477 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.182849 (+ 8372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.183736 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.184936 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.185265 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.185348 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:19.187791 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:19.187795 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:19.187798 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:19.187833 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:19.189740 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:19.190303 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:19.190307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:19.190311 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:19.190429 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:19.190761 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:19.190765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:19.190767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:19.191006 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:19.191132 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:19.191572 (+ 440us) server_negotiation.cc:299] Negotiation successful
0619 02:34:19.191722 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481}
W20260619 02:34:19.196125 8238 catalog_manager.cc:1558] T 00000000000000000000000000000000 P af2905691054452ab6410fa1c440507b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:34:19.196310 8238 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:34:19.199183 8109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:19.172087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60326 (local address 127.17.96.253:42311)
0619 02:34:19.172203 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.172208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.183547 (+ 11339us) server_negotiation.cc:407] Connection header received
0619 02:34:19.183722 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.183726 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.183781 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.183833 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.184799 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.185592 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.192140 (+ 6548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.192505 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.192636 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:19.194313 (+ 1677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:19.194318 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:19.194321 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:19.194352 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:19.196261 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:19.196974 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:19.196989 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:19.196991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:19.197082 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:19.197431 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:19.197435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:19.197436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:19.197692 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:19.197876 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:19.198861 (+ 985us) server_negotiation.cc:299] Negotiation successful
0619 02:34:19.199015 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 02:34:19.203855 8185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:19.171337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56988 (local address 127.17.96.252:35977)
0619 02:34:19.171459 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.171463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.182118 (+ 10655us) server_negotiation.cc:407] Connection header received
0619 02:34:19.182260 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.182263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.182321 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.182421 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.183335 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.184060 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.186141 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:19.186474 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:19.186552 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:19.196842 (+ 10290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:34:19.196846 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:34:19.196849 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:34:19.196878 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:34:19.198614 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:19.202258 (+ 3644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:19.202263 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:19.202265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:19.202362 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:34:19.202686 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:34:19.202689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:34:19.202691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:34:19.202931 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:34:19.203081 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:19.203464 (+ 383us) server_negotiation.cc:299] Negotiation successful
0619 02:34:19.203639 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 02:34:19.205240 8227 sys_catalog.cc:455] T 00000000000000000000000000000000 P af2905691054452ab6410fa1c440507b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ad273cbbd8448f9ac746777e5330d2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44767 } } peers { permanent_uuid: "61cbba2b62ef4ebd8bce5f1be99b4753" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } } peers { permanent_uuid: "af2905691054452ab6410fa1c440507b" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } } }
I20260619 02:34:19.205507 8227 sys_catalog.cc:458] T 00000000000000000000000000000000 P af2905691054452ab6410fa1c440507b [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:34:19.206082 8154 raft_consensus.cc:1261] T 00000000000000000000000000000000 P af2905691054452ab6410fa1c440507b [term 1 FOLLOWER]: Refusing update from remote peer 33ad273cbbd8448f9ac746777e5330d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:34:19.206410 8077 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 61cbba2b62ef4ebd8bce5f1be99b4753 [term 1 FOLLOWER]: Refusing update from remote peer 33ad273cbbd8448f9ac746777e5330d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:34:19.207010 8222 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af2905691054452ab6410fa1c440507b" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:34:19.207785 8222 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61cbba2b62ef4ebd8bce5f1be99b4753" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:34:19.211869 8201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61cbba2b62ef4ebd8bce5f1be99b4753 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33ad273cbbd8448f9ac746777e5330d2. Latest consensus state: current_term: 1 leader_uuid: "33ad273cbbd8448f9ac746777e5330d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ad273cbbd8448f9ac746777e5330d2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44767 } } peers { permanent_uuid: "61cbba2b62ef4ebd8bce5f1be99b4753" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } } peers { permanent_uuid: "af2905691054452ab6410fa1c440507b" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } } }
I20260619 02:34:19.212021 8201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61cbba2b62ef4ebd8bce5f1be99b4753 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:34:19.219183 8227 sys_catalog.cc:455] T 00000000000000000000000000000000 P af2905691054452ab6410fa1c440507b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33ad273cbbd8448f9ac746777e5330d2. Latest consensus state: current_term: 1 leader_uuid: "33ad273cbbd8448f9ac746777e5330d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ad273cbbd8448f9ac746777e5330d2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44767 } } peers { permanent_uuid: "61cbba2b62ef4ebd8bce5f1be99b4753" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } } peers { permanent_uuid: "af2905691054452ab6410fa1c440507b" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } } }
I20260619 02:34:19.219352 8227 sys_catalog.cc:458] T 00000000000000000000000000000000 P af2905691054452ab6410fa1c440507b [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:34:19.231941 8222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "33ad273cbbd8448f9ac746777e5330d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ad273cbbd8448f9ac746777e5330d2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44767 } } peers { permanent_uuid: "61cbba2b62ef4ebd8bce5f1be99b4753" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } } peers { permanent_uuid: "af2905691054452ab6410fa1c440507b" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } } }
I20260619 02:34:19.232146 8222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:34:19.232278 8226 mvcc.cc:205] Tried to move back new op lower bound from 7298402136898785280 to 7298402136774103040. Current Snapshot: MvccSnapshot[applied={T|T < 7298402136898785280}]
I20260619 02:34:19.236076 8201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61cbba2b62ef4ebd8bce5f1be99b4753 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "33ad273cbbd8448f9ac746777e5330d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ad273cbbd8448f9ac746777e5330d2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44767 } } peers { permanent_uuid: "61cbba2b62ef4ebd8bce5f1be99b4753" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } } peers { permanent_uuid: "af2905691054452ab6410fa1c440507b" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } } }
I20260619 02:34:19.236261 8201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61cbba2b62ef4ebd8bce5f1be99b4753 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:34:19.238230 8189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "33ad273cbbd8448f9ac746777e5330d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ad273cbbd8448f9ac746777e5330d2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44767 } } peers { permanent_uuid: "61cbba2b62ef4ebd8bce5f1be99b4753" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } } peers { permanent_uuid: "af2905691054452ab6410fa1c440507b" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } } }
I20260619 02:34:19.238386 8189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:34:19.239715 8227 sys_catalog.cc:455] T 00000000000000000000000000000000 P af2905691054452ab6410fa1c440507b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "33ad273cbbd8448f9ac746777e5330d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33ad273cbbd8448f9ac746777e5330d2" member_type: VOTER last_known_addr { host: "127.17.96.254" port: 44767 } } peers { permanent_uuid: "61cbba2b62ef4ebd8bce5f1be99b4753" member_type: VOTER last_known_addr { host: "127.17.96.253" port: 42311 } } peers { permanent_uuid: "af2905691054452ab6410fa1c440507b" member_type: VOTER last_known_addr { host: "127.17.96.252" port: 35977 } } }
I20260619 02:34:19.239853 8227 sys_catalog.cc:458] T 00000000000000000000000000000000 P af2905691054452ab6410fa1c440507b [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:34:19.240697 8242 mvcc.cc:205] Tried to move back new op lower bound from 7298402136898785280 to 7298402136774103040. Current Snapshot: MvccSnapshot[applied={T|T < 7298402136898785280}]
I20260619 02:34:19.240904 8240 mvcc.cc:205] Tried to move back new op lower bound from 7298402136898785280 to 7298402136774103040. Current Snapshot: MvccSnapshot[applied={T|T < 7298402136898785280}]
I20260619 02:34:19.249421 8225 catalog_manager.cc:1347] Generated new cluster ID: c6629421655543d5b781b818fc1ea6d3
I20260619 02:34:19.249521 8225 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:34:19.277592 8225 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:34:19.279100 8225 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:34:19.306020 8225 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33ad273cbbd8448f9ac746777e5330d2: Generated new TSK 0
I20260619 02:34:19.307063 8225 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:34:19.338268 8185 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:19.336000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57002 (local address 127.17.96.252:35977)
0619 02:34:19.336144 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.336149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.337135 (+ 986us) server_negotiation.cc:407] Connection header received
0619 02:34:19.337332 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.337336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.337387 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.337440 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.338123 (+ 683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57002: BlockingRecv error: recv got EOF from 127.0.0.1:57002 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260619 02:34:19.339890 8109 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:19.336508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60334 (local address 127.17.96.253:42311)
0619 02:34:19.336623 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.336627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.338637 (+ 2010us) server_negotiation.cc:407] Connection header received
0619 02:34:19.339062 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.339067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.339121 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.339176 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.339749 (+ 573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60334: BlockingRecv error: recv got EOF from 127.0.0.1:60334 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260619 02:34:19.341988 8032 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:19.340402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58936 (local address 127.17.96.254:44767)
0619 02:34:19.340515 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.340521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.341034 (+ 513us) server_negotiation.cc:407] Connection header received
0619 02:34:19.341227 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.341230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.341280 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.341331 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.341862 (+ 531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58936: BlockingRecv error: recv got EOF from 127.0.0.1:58936 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260619 02:34:19.346889 8109 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:19.345142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60338 (local address 127.17.96.253:42311)
0619 02:34:19.345289 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.345293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.345834 (+ 541us) server_negotiation.cc:407] Connection header received
0619 02:34:19.346081 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.346085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.346163 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.346221 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.346772 (+ 551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60338: BlockingRecv error: recv got EOF from 127.0.0.1:60338 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260619 02:34:19.352222 8109 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:19.350488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60354 (local address 127.17.96.253:42311)
0619 02:34:19.350608 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.350612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.351197 (+ 585us) server_negotiation.cc:407] Connection header received
0619 02:34:19.351410 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.351414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.351473 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.351529 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.352104 (+ 575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60354: BlockingRecv error: recv got EOF from 127.0.0.1:60354 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260619 02:34:19.358487 8109 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:19.356544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60364 (local address 127.17.96.253:42311)
0619 02:34:19.356663 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.356667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.357322 (+ 655us) server_negotiation.cc:407] Connection header received
0619 02:34:19.357564 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.357568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.357635 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.357695 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.358356 (+ 661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60364: BlockingRecv error: recv got EOF from 127.0.0.1:60364 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260619 02:34:19.367029 8109 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:19.365110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60372 (local address 127.17.96.253:42311)
0619 02:34:19.365295 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.365302 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.365947 (+ 645us) server_negotiation.cc:407] Connection header received
0619 02:34:19.366194 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.366198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.366257 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.366316 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.366908 (+ 592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60372: BlockingRecv error: recv got EOF from 127.0.0.1:60372 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260619 02:34:19.379788 8109 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:19.377793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60376 (local address 127.17.96.253:42311)
0619 02:34:19.377935 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.377940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.378619 (+ 679us) server_negotiation.cc:407] Connection header received
0619 02:34:19.378866 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.378870 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.378932 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.378992 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.379660 (+ 668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60376: BlockingRecv error: recv got EOF from 127.0.0.1:60376 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260619 02:34:19.400588 8109 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:19.398681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60390 (local address 127.17.96.253:42311)
0619 02:34:19.398808 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.398813 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.399448 (+ 635us) server_negotiation.cc:407] Connection header received
0619 02:34:19.399657 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.399662 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.399719 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.399772 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.400456 (+ 684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60390: BlockingRecv error: recv got EOF from 127.0.0.1:60390 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260619 02:34:19.436219 8109 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:19.435140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60392 (local address 127.17.96.253:42311)
0619 02:34:19.435315 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.435318 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.435433 (+ 115us) server_negotiation.cc:407] Connection header received
0619 02:34:19.435532 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.435535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.435583 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.435670 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.436119 (+ 449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60392: BlockingRecv error: recv got EOF from 127.0.0.1:60392 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260619 02:34:19.504187 8109 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:19.502908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60404 (local address 127.17.96.253:42311)
0619 02:34:19.503092 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.503097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.503257 (+ 160us) server_negotiation.cc:407] Connection header received
0619 02:34:19.503437 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.503441 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.503497 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.503600 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.504066 (+ 466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60404: BlockingRecv error: recv got EOF from 127.0.0.1:60404 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260619 02:34:19.635689 8109 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:19.634572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60414 (local address 127.17.96.253:42311)
0619 02:34:19.634754 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.634758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.634776 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:34:19.634941 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.634944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.635006 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.635108 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.635537 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60414: BlockingRecv error: recv got EOF from 127.0.0.1:60414 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260619 02:34:19.895283 8109 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:34:19.893882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60420 (local address 127.17.96.253:42311)
0619 02:34:19.894161 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:19.894168 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:19.894402 (+ 234us) server_negotiation.cc:407] Connection header received
0619 02:34:19.894519 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:19.894522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:19.894584 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:19.894659 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:34:19.895132 (+ 473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60420: BlockingRecv error: recv got EOF from 127.0.0.1:60420 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
I20260619 02:34:20.165998 8212 catalog_manager.cc:1259] Loaded cluster ID: c6629421655543d5b781b818fc1ea6d3
I20260619 02:34:20.166085 8212 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 61cbba2b62ef4ebd8bce5f1be99b4753: loading cluster ID for follower catalog manager: success
I20260619 02:34:20.167814 8212 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 61cbba2b62ef4ebd8bce5f1be99b4753: acquiring CA information for follower catalog manager: success
I20260619 02:34:20.169116 8212 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 61cbba2b62ef4ebd8bce5f1be99b4753: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:34:20.197722 8238 catalog_manager.cc:1259] Loaded cluster ID: c6629421655543d5b781b818fc1ea6d3
I20260619 02:34:20.197882 8238 catalog_manager.cc:1552] T 00000000000000000000000000000000 P af2905691054452ab6410fa1c440507b: loading cluster ID for follower catalog manager: success
I20260619 02:34:20.199574 8238 catalog_manager.cc:1574] T 00000000000000000000000000000000 P af2905691054452ab6410fa1c440507b: acquiring CA information for follower catalog manager: success
I20260619 02:34:20.200755 8238 catalog_manager.cc:1602] T 00000000000000000000000000000000 P af2905691054452ab6410fa1c440507b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:34:20.413578 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:20.409552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60434 (local address 127.17.96.253:42311)
0619 02:34:20.409849 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:20.409853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:20.409944 (+ 91us) server_negotiation.cc:407] Connection header received
0619 02:34:20.410064 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:20.410067 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:20.410195 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:20.410302 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:34:20.410544 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:20.411093 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:20.412037 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:20.412288 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:20.412382 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:20.412671 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:34:20.413147 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:34:20.413242 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:20.413297 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 02:34:20.413353 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":98,"threads_started":1}
I20260619 02:34:20.420567 8316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:20.416820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57006 (local address 127.17.96.252:35977)
0619 02:34:20.417047 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:20.417065 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:20.417078 (+ 13us) server_negotiation.cc:407] Connection header received
0619 02:34:20.417163 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:20.417165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:20.417286 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:20.417388 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:34:20.417587 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:20.418178 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:20.419002 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:20.419248 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:20.419323 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:20.419668 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:34:20.420130 (+ 462us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:34:20.420222 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:20.420303 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 02:34:20.420357 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":72,"threads_started":1}
I20260619 02:34:20.427989 8316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:20.424046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57018 (local address 127.17.96.252:35977)
0619 02:34:20.424162 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:20.424167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:20.424373 (+ 206us) server_negotiation.cc:407] Connection header received
0619 02:34:20.424561 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:20.424565 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:20.424614 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:20.424686 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:34:20.425101 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:20.425880 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:20.426799 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:20.427026 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:20.427143 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:20.427471 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:34:20.427581 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:34:20.427658 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:20.427770 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 02:34:20.427838 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 02:34:20.428681 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:20.423963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60442 (local address 127.17.96.253:42311)
0619 02:34:20.424101 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:20.424106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:20.424361 (+ 255us) server_negotiation.cc:407] Connection header received
0619 02:34:20.424459 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:20.424466 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:20.424540 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:20.424624 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:34:20.425005 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:20.425903 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:20.427124 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:20.427450 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:20.427536 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:20.427855 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:34:20.427936 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:34:20.427981 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:20.428444 (+ 463us) server_negotiation.cc:299] Negotiation successful
0619 02:34:20.428507 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:34:20.429436 8324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:34:20.423783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58952 (local address 127.17.96.254:44767)
0619 02:34:20.424045 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0619 02:34:20.424050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:34:20.424066 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:34:20.424361 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:34:20.424365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:34:20.424572 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:34:20.424698 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:34:20.425020 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:20.425690 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:20.427599 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:34:20.427963 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:34:20.428082 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:34:20.428471 (+ 389us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:34:20.429009 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:34:20.429134 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 02:34:20.429216 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 02:34:20.429276 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":88,"threads_started":1}
I20260619 02:34:20.432531 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 7960
I20260619 02:34:20.439054 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 8035
I20260619 02:34:20.445161 17795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskhpFBxW/build/debug/bin/kudu with pid 8110
2026-06-19T02:34:20Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3156 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6346 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (298432 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST